CN112259116A - A noise reduction method, device, electronic device and storage medium for audio data - Google Patents

Abstract

Translated fromChinese

本公开实施例公开了一种音频数据的降噪方法、装置、电子设备及存储介质，其中该方法包括：对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。通过基于降噪模型对第一子带降噪，通过增益映射对其他子带降噪，不仅能够提高模型训练效率，也能够提高音频数据的降噪效率。

Embodiments of the present disclosure disclose a noise reduction method, device, electronic device, and storage medium for audio data, wherein the method includes: performing subband processing on audio frame data, wherein the frequency intervals of each subband are different, and the interval is the largest The subband corresponding to the frequency interval with the smallest value is regarded as the first subband; the frequency domain information of the first subband is input into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband, and the Gain of a subband; based on the gain of the first subband, determine the gains of other subbands except the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands; The frequency domain information after noise reduction and the result of performing noise reduction processing on other subbands determine the audio frame data after noise reduction. By denoising the first subband based on the noise reduction model and denoising other subbands through gain mapping, not only the model training efficiency can be improved, but also the noise reduction efficiency of the audio data can be improved.

Description

Translated fromChinese

一种音频数据的降噪方法、装置、电子设备及存储介质A noise reduction method, device, electronic device and storage medium for audio data

技术领域technical field

本公开实施例涉及计算机技术领域，尤其涉及一种音频数据的降噪方法、装置、电子设备及存储介质。The embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a noise reduction method, apparatus, electronic device, and storage medium for audio data.

背景技术Background technique

近年来，利用深度学习模型来进行音频数据降噪是降噪的主流趋势之一。可利用音频数据样本进行深度学习模型的训练，且利用低频采样率的音频数据样本训练得到的模型，可应用于低频采样率的音频数据降噪，利用高频采样率的音频数据样本训练得到的模型，可应用于高频采样率的音频数据降噪。In recent years, the use of deep learning models to denoise audio data is one of the mainstream trends in noise reduction. The deep learning model can be trained with audio data samples, and the model obtained by training with the audio data samples of the low-frequency sampling rate can be applied to the noise reduction of the audio data of the low-frequency sampling rate, and the model obtained by training the audio data samples of the high-frequency sampling rate A model that can be applied to noise reduction of audio data at high sampling rates.

现有降噪方法至少包括如下技术问题：当音频数据样本的采样率越高时，模型训练需要更长的样本点数，更多的模型参数和更复杂的模型结构，导致模型训练效率较低；并且，当基于该模型进行音频数据降噪时，降噪处理也会相应耗时较长，导致音频数据降噪效率较低。Existing noise reduction methods include at least the following technical problems: when the sampling rate of audio data samples is higher, model training requires longer sample points, more model parameters and more complex model structure, resulting in lower model training efficiency; Moreover, when noise reduction of audio data is performed based on this model, the noise reduction processing will also take a long time, resulting in low noise reduction efficiency of audio data.

发明内容SUMMARY OF THE INVENTION

本公开实施例提供一种音频数据的降噪方法、装置、电子设备及存储介质，不仅能够提高模型训练效率，也能够提高音频数据的降噪效率。Embodiments of the present disclosure provide a noise reduction method, device, electronic device, and storage medium for audio data, which can not only improve model training efficiency, but also improve the noise reduction efficiency of audio data.

第一方面，本公开实施例提供了一种音频数据的降噪方法，包括：In a first aspect, an embodiment of the present disclosure provides a noise reduction method for audio data, including:

对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；The audio frame data is subjected to molecular band processing, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband;

将所述第一子带的频域信息输入降噪模型，以使所述降噪模型输出所述第一子带降噪后的频域信息，和所述第一子带的增益；inputting the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the frequency domain information after the noise reduction of the first subband and the gain of the first subband;

基于所述第一子带的增益，确定除所述第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；Based on the gain of the first subband, determine the gains of other subbands except the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands;

根据所述第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。The denoised audio frame data is determined according to the denoised frequency domain information of the first subband and the result of performing denoising processing on other subbands.

第二方面，本公开实施例还提供了一种音频数据的降噪装置，包括：In a second aspect, an embodiment of the present disclosure further provides a noise reduction device for audio data, including:

分子带模块，用于对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；The molecular band module is used to perform molecular band processing on the audio frame data, wherein the frequency intervals of each sub-band are different, and the sub-band corresponding to the frequency interval with the smallest interval maximum value is used as the first sub-band;

第一降噪模块，用于将所述第一子带的频域信息输入降噪模型，以使所述降噪模型输出所述第一子带降噪后的频域信息，和所述第一子带的增益；A first noise reduction module, configured to input the frequency domain information of the first subband into a noise reduction model, so that the noise reduction model outputs the frequency domain information after noise reduction of the first subband, and the A subband gain;

第二降噪模块，用于基于所述第一子带的增益，确定除所述第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；A second noise reduction module, configured to determine the gains of other subbands except the first subband based on the gain of the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands ;

降噪数据确定模块，用于根据所述第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。The noise reduction data determination module is configured to determine the audio frame data after noise reduction according to the frequency domain information after noise reduction of the first subband and the result of performing noise reduction processing on other subbands.

第三方面，本公开实施例还提供了一种电子设备，所述电子设备包括：In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

一个或多个处理器；one or more processors;

存储装置，用于存储一个或多个程序，storage means for storing one or more programs,

当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本公开实施例任一所述的音频数据的降噪方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the noise reduction method for audio data according to any one of the embodiments of the present disclosure.

第四方面，本公开实施例还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如本公开实施例任一所述的音频数据的降噪方法。In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, when executed by a computer processor, the computer-executable instructions are used to execute the audio data according to any one of the embodiments of the present disclosure noise reduction method.

本公开实施例的技术方案，对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。The technical solution of the embodiment of the present disclosure is to perform molecular band processing on the audio frame data, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband; the first subband The frequency domain information of the band is input into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband and the gain of the first subband; Gains of other subbands except for the other subbands, and noise reduction processing is performed on the other subbands according to the gains of the other subbands; Determines the denoised audio frame data.

人耳最敏感的声音频率范围，通常认为是较低频率的范围。基于此，本公开实施例的技术方案中，把分帧后的音频数据划分为各个子带，通过降噪模型只对低频区间对应的第一子带进行降噪，并根据降噪模型输出的第一子带的增益，对其他子带的增益进行映射，以对其他子带进行降噪。相较于传统的利用降噪模型对全频带降噪来说，减少了整体的降噪耗时，提高了降噪效率。The range of sound frequencies to which the human ear is most sensitive, generally considered to be the lower frequency range. Based on this, in the technical solution of the embodiment of the present disclosure, the divided audio data is divided into sub-bands, and only the first sub-band corresponding to the low-frequency interval is noise-reduced by the noise-reduction model. The gain of the first subband is mapped to the gains of other subbands to denoise the other subbands. Compared with the traditional noise reduction model for full-band noise reduction, the overall noise reduction time is reduced and the noise reduction efficiency is improved.

此外，由于降噪模型只应用于第一子带降噪，而非全频带降噪，降噪模型可以只通第一子带的样本数据进行训练，从而能够提高模型训练效率。尤其针对高采样率的音频数据进行模型训练的情况，通过只对第一子带的样本数据进行训练，可大大减少模型参数和模型复杂度，提高模型训练效率。In addition, since the noise reduction model is only applied to the first sub-band noise reduction, not the full-band noise reduction, the noise reduction model can be trained only through the sample data of the first sub-band, thereby improving the model training efficiency. Especially in the case of model training with high sampling rate audio data, by training only the sample data of the first subband, model parameters and model complexity can be greatly reduced, and model training efficiency can be improved.

附图说明Description of drawings

结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent when taken in conjunction with the accompanying drawings and with reference to the following detailed description. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

图1为本公开实施例一所提供的一种音频数据的降噪方法的流程示意图；FIG. 1 is a schematic flowchart of a noise reduction method for audio data according to Embodiment 1 of the present disclosure;

图2为本公开实施例二所提供的一种音频数据的降噪方法的流程示意图；FIG. 2 is a schematic flowchart of a noise reduction method for audio data according to Embodiment 2 of the present disclosure;

图3为本公开实施例三所提供的一种音频数据的降噪方法的流程示意图；3 is a schematic flowchart of a noise reduction method for audio data according to Embodiment 3 of the present disclosure;

图4为本公开实施例五所提供的一种音频数据的降噪装置结构示意图；FIG. 4 is a schematic structural diagram of a noise reduction device for audio data according to Embodiment 5 of the present disclosure;

图5为本公开实施例六所提供的一种电子设备结构示意图。FIG. 5 is a schematic structural diagram of an electronic device according to Embodiment 6 of the present disclosure.

具体实施方式Detailed ways

下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for the purpose of A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the protection scope of the present disclosure.

应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this regard.

本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。As used herein, the term "including" and variations thereof are open-ended inclusions, ie, "including but not limited to". The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below.

需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or interdependence.

需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。It should be noted that the modifications of "a" and "a plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, they should be understood as "one or a plurality of". multiple".

实施例一Example 1

图1为本公开实施例一所提供的一种音频数据的降噪方法的流程示意图，本公开实施例适用于音频数据降噪的情形，尤其适用于高采样率的音频数据的实时降噪的情形。该方法可以由音频数据的降噪装置来执行，该装置可以通过软件和/或硬件的形式实现，该装置可配置于电子设备中，例如配置于计算机中。FIG. 1 is a schematic flowchart of a method for noise reduction of audio data provided by the first embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation of noise reduction of audio data, and is especially suitable for the real-time noise reduction of audio data with high sampling rate. situation. The method may be performed by a noise reduction apparatus for audio data, the apparatus may be implemented in the form of software and/or hardware, and the apparatus may be configured in an electronic device, such as a computer.

如图1所示，本实施例提供的音频数据的降噪方法，包括：As shown in FIG. 1, the noise reduction method for audio data provided by this embodiment includes:

S110、对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带。S110. Perform molecular band processing on the audio frame data, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband.

本公开实施例中，需要降噪的音频数据可以是实时采集的音频数据流，也可以是采集完毕的音频数据文件。In the embodiment of the present disclosure, the audio data to be noise-reduced may be an audio data stream collected in real time, or may be an audio data file that has been collected.

通常在对音频数据进行降噪前，可以对音频数据进行分帧处理，得到音频帧数据。其中，对音频数据进行分帧处理，可以理解为将音频数据中的采样点进行分段处理，且可以基于现有的Matlab分帧代码或其他分帧方式进行分帧，在此不做穷举。通过对音频数据进行分帧，不仅有利于音频数据流的实时处理，无需等到音频数据采集完毕再降噪，还有利于音频数据文件中各音频帧数据进行同时降噪，以提高降噪效率。Usually, before noise reduction is performed on the audio data, the audio data can be processed in frames to obtain audio frame data. Among them, the framing processing of the audio data can be understood as the framing processing of the sampling points in the audio data, and the framing can be carried out based on the existing Matlab framing code or other framing methods, which will not be exhaustive here. . Framing the audio data is not only conducive to the real-time processing of the audio data stream, without waiting for the audio data to be collected before noise reduction, but also conducive to simultaneous noise reduction of each audio frame data in the audio data file to improve the noise reduction efficiency.

在获取到音频帧数据时，可以对每帧数据进行分子带处理。其中，音频帧数据的不同子带可以认为是，音频帧数据中包含的不同频率区间的数据部分。其中，可以基于时域和/或频域的处理方法，将音频帧数据解析为不同频率区间的数据部分。When the audio frame data is acquired, molecular band processing can be performed on each frame data. Wherein, the different subbands of the audio frame data can be considered as data parts of different frequency intervals included in the audio frame data. Wherein, the audio frame data may be parsed into data parts of different frequency intervals based on the processing method in the time domain and/or the frequency domain.

其中，音频数据的每个子带的频率区间不同，可以理解为每个子带的频率区间的端点值不完全一致。各子带的频率区间可以呈现重叠、依次相邻或不相邻等情况。示例性的，采样率为48khz的音频帧数据，其频率范围为0hz-24khz，该音频帧数据的各子带对应的频率区间，可以是重叠的情况，例如是0hz-8khz、7khz-16khz和15khz-24khz；也可以是依次相邻的情况，例如是0hz-8khz、8khz-16khz和16khz-24khz；还可以是不相邻的情况，例如是0hz-8khz、9khz-16khz和17khz-24khz等。Wherein, the frequency intervals of each subband of the audio data are different, which can be understood as the endpoint values of the frequency intervals of each subband are not completely consistent. The frequency intervals of each subband may overlap, be adjacent in sequence, or not be adjacent to each other. Exemplarily, the audio frame data with a sampling rate of 48khz has a frequency range of 0hz-24khz, and the frequency interval corresponding to each subband of the audio frame data may be overlapping, for example, 0hz-8khz, 7khz-16khz and 15khz-24khz; it can also be adjacent in sequence, such as 0hz-8khz, 8khz-16khz and 16khz-24khz; it can also be non-adjacent, such as 0hz-8khz, 9khz-16khz and 17khz-24khz, etc. .

在本公开实施例一些优选的实现方式中，音频帧数据的各子带的频率区间依次相邻，可以认为前一个子带的频率区间的右侧端点值，等于后一个子带的频域区间的左侧端点值。在这些优选的实现方式中，通过将各子带的频率区间相邻设置，不仅避免了重叠频率区域的音频帧数据重复降噪处理，还可以在降噪时不遗漏一些频率区域的音频帧数据，保证音频帧数据的全带都可以完成降噪，在一定程度上可提升降噪效果。In some preferred implementations of the embodiments of the present disclosure, the frequency intervals of each subband of the audio frame data are adjacent in sequence, and it can be considered that the right end value of the frequency interval of the previous subband is equal to the frequency domain interval of the next subband the left endpoint value of . In these preferred implementations, by arranging the frequency intervals of each subband adjacent to each other, not only the repeated noise reduction processing of the audio frame data in the overlapping frequency regions is avoided, but also the audio frame data in some frequency regions can not be omitted during noise reduction. , to ensure that the entire band of audio frame data can complete noise reduction, which can improve the noise reduction effect to a certain extent.

其中，区间最大值最小的频率区间，可以认为是各子带对应的频率区间中，相对的低频区间。此外，频率区间中相对的低频区间，还可以用区间最小值最小的频率区间，或者区间中间值最小的频率区间等描述方式来表示，在此不做穷举。Among them, the frequency interval with the smallest interval maximum value can be regarded as the relative low frequency interval in the frequency interval corresponding to each subband. In addition, the relative low-frequency interval in the frequency interval may also be represented by a description method such as a frequency interval with the smallest interval minimum value, or a frequency interval with the smallest interval median value, etc., which is not exhaustive here.

人耳最敏感的声音频率范围为200hz-800hz，与声音频率20hz-20000hz范围相比，通常认为是较低频率的范围。因此，将音频帧数据分子带后，通过将区间最大值最小的频率区间对应的子带，作为第一子带，旨在从各子带中选取出包含人耳敏感的频率范围的，相对低频区间对应的第一子带，为后续通过降噪模型对第一子带降噪奠定基础。The human ear is most sensitive to the sound frequency range of 200hz-800hz, which is generally considered to be a lower frequency range compared to the sound frequency range of 20hz-20000hz. Therefore, after the audio frame data is sub-banded, the sub-band corresponding to the frequency interval with the smallest interval maximum value is taken as the first sub-band, and the purpose is to select the frequency range that contains the human ear sensitivity from each sub-band. The first subband corresponding to the interval lays the foundation for the subsequent noise reduction of the first subband through the noise reduction model.

S120、将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益。S120. Input the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband and the gain of the first subband.

本公开实施例中，降噪模型例如可以是，循环神经网络噪声抑制(RecurrentNeural Network Noise，RNNoise)模型，卷积神经网络(Convolutional Neural Networks，CNN)模型等，其他音频降噪模型也可应用于此，在此不做穷举。降噪模型可以通过预先训练得到，且基于降噪模型可以实现对音频帧数据的第一子带的降噪，同时可以输出第一子带的增益。In the embodiment of the present disclosure, the noise reduction model may be, for example, a Recurrent Neural Network Noise (RNNoise) model, a Convolutional Neural Networks (CNN) model, etc. Other audio noise reduction models may also be applied to Therefore, it will not be exhaustive here. The noise reduction model can be obtained by pre-training, and based on the noise reduction model, noise reduction of the first subband of the audio frame data can be realized, and the gain of the first subband can be output at the same time.

其中，对于每帧音频数据来说，第一子带的增益可以认为是，第一子带对应的频率区间中每个频点对应的增益的集合，且每个频点对应的增益为一个增益数值。其中，增益数值例如可以是降噪后的频域信息，占降噪前的频域信息的占比数值。其中，增益通常在0-1范围内，且增益越趋近于0，可认为该音频帧数据越趋近于噪声，越需要对该音频帧数据进行消除。Among them, for each frame of audio data, the gain of the first subband can be considered as a set of gains corresponding to each frequency point in the frequency interval corresponding to the first subband, and the gain corresponding to each frequency point is a gain numerical value. Wherein, the gain value may be, for example, the frequency domain information after noise reduction, and the ratio value of the frequency domain information before noise reduction. Wherein, the gain is usually in the range of 0-1, and the closer the gain is to 0, it can be considered that the audio frame data is closer to noise, and the audio frame data needs to be eliminated more.

与传统的对音频数据的全带数据进行降噪处理相比，本公开实施例通过利用降噪模型只针对第一子带进行降噪处理，可以减少音频数据的整体降噪耗时，提高了降噪效率，在音频数据实时降噪的应用中优势非常明显。Compared with the traditional noise reduction processing on the full-band data of the audio data, the embodiment of the present disclosure can reduce the overall noise reduction time of the audio data by using the noise reduction model to perform the noise reduction processing only on the first subband, and improve the performance of the noise reduction. Noise reduction efficiency has obvious advantages in the application of real-time noise reduction of audio data.

并且，通常采样率越高，传统的降噪模型的结构越复杂，降噪过程耗费的计算资源越多，致使音频数据降噪很难落地到笔记本或手机等计算资源有限的终端中，不利于实际工程的应用。而本公开实施例提供的降噪模型，对高采样率的音频数据降噪时，可以只对其低频区间的第一子带进行降噪，因此模型复杂度大大降低，耗费的计算资源也大大减小，有利于落地到计算资源有限的终端中去，有利于实际工程的应用。In addition, the higher the sampling rate, the more complex the structure of the traditional noise reduction model, and the more computing resources the noise reduction process consumes, making it difficult for audio data noise reduction to be implemented in terminals with limited computing resources such as laptops or mobile phones, which is not conducive to Practical engineering applications. However, in the noise reduction model provided by the embodiment of the present disclosure, when denoising audio data with a high sampling rate, noise reduction can be performed only on the first subband in the low frequency range, so the model complexity is greatly reduced, and the computational resources consumed are also greatly reduced. It is beneficial to land in terminals with limited computing resources, and is beneficial to the application of practical engineering.

在本公开实施例一些可选的实现方式中，降噪模型的训练方式，包括：获取与第一子带的频率区间相同的样本帧数据，以及样本帧数据的目标频域信息；利用样本帧数据的频域信息，以及目标频域信息，对降噪模型进行训练，直至降噪模型收敛。In some optional implementations of the embodiments of the present disclosure, the training method of the noise reduction model includes: obtaining sample frame data that is the same as the frequency interval of the first subband, and target frequency domain information of the sample frame data; using the sample frame data The frequency domain information of the data and the target frequency domain information are used to train the noise reduction model until the noise reduction model converges.

其中，在进行降噪模型训练时，所需要的样本数据，可以与需要降噪的音频数据的频率范围相同，也可以与需要降噪的音频数据的频率范围不同，只需保证样本数据经过分帧，和/或分子带处理后，样本帧数据与第一子带具备相同的频率区间即可。Among them, when training the noise reduction model, the required sample data can be the same as the frequency range of the audio data that needs to be denoised, or it can be different from the frequency range of the audio data that needs to be denoised. After frame and/or molecular band processing, the sample frame data and the first subband only need to have the same frequency interval.

通常，音频数据的采样率需达到其频率最大值的两倍，才能满足采集需求。现有大部分开源的音频数据集的采样率较低，例如采样率为16khz。当需要降噪的音频数据的频率最大值较高时，其采样率也较高，例如采样率达到32khz、48khz等，此时若采用传统的利用全带样本数据进行降噪模型训练，不仅没有开源的音频数据集支持，需要耗费大量时间进行高采样率的样本数据采集，而且训练过程必然需要更多的样本点数，更多的模型参数和更复杂的模型结构，这将导致模型训练效率较低。Typically, audio data needs to be sampled at twice its maximum frequency to meet acquisition needs. Most of the existing open source audio datasets have lower sampling rates, such as 16khz. When the maximum frequency of the audio data that needs noise reduction is high, the sampling rate is also high, for example, the sampling rate reaches 32khz, 48khz, etc. At this time, if the traditional noise reduction model training using the full-band sample data is used, not only there is no Open source audio data set support requires a lot of time to collect sample data with high sampling rate, and the training process must require more sample points, more model parameters and more complex model structure, which will lead to higher model training efficiency. Low.

而本公开这些可选的实施方式中，可以利用现有的采样率较低的开源音频数据集，将数据集中的样本数据进行分帧，和/或分子带处理，以使样本帧数据与第一子带具备相同的频率区间，就可以进行降噪模型训练，以应用于高采样率的音频数据的降噪。从而，不仅无需耗费时间进行高采样率的样本数据采集，而且只根据第一子带的频率区间的样本帧数据进行训练，可大大减少模型参数和模型复杂度，提高模型训练效率。In these optional embodiments of the present disclosure, an existing open source audio data set with a lower sampling rate may be used to divide the sample data in the data set into frames, and/or perform molecular band processing, so that the sample frame data and the first Once the subbands have the same frequency range, a noise reduction model can be trained to apply noise reduction to high sampling rate audio data. Therefore, not only does it not take time to collect sample data with a high sampling rate, but also the training is only performed according to the sample frame data in the frequency range of the first subband, which can greatly reduce model parameters and model complexity, and improve model training efficiency.

S130、基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理。S130. Based on the gain of the first subband, determine the gains of other subbands except the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands.

本公开实施例中，由于各子带的频率区间依次相邻，且相邻频点增益的关联性较强，可基于第一子带的增益，对其他子带对应的频率区间中，各频点的增益进行确定。其中，基于第一子带的增益，确定除第一子带外的其他子带的增益的方式，例如可以是按照相邻频点间的增益映射关系，依次确定其他子带的各频点的增益，或者可以是根据第一子带的频率区间中频率值较大的频点的增益，确定其他子带的增益等。In this embodiment of the present disclosure, since the frequency intervals of each subband are adjacent in sequence, and the correlation between the gains of adjacent frequency points is strong, based on the gain of the first subband, the frequency intervals of each frequency interval corresponding to the other subbands may be point gain is determined. Wherein, based on the gain of the first subband, the method of determining the gain of other subbands except the first subband may be, for example, according to the gain mapping relationship between adjacent frequency points, sequentially determining the gain of each frequency point of the other subbands The gain, or the gain of other subbands may be determined according to the gain of a frequency point with a larger frequency value in the frequency interval of the first subband, and the like.

在本公开实施例一些可选的实现方式中，基于第一子带的增益，确定除第一子带外的其他子带的增益，包括：综合第一子带的频率区间中，预设频点值到区间最大值的平均增益，以及降噪模型输出的第一子带属于预设类别的判决概率，确定除第一子带外的其他子带的增益。In some optional implementation manners of the embodiments of the present disclosure, determining the gain of other subbands except the first subband based on the gain of the first subband includes: synthesizing the frequency interval of the first subband, a preset frequency The average gain from the point value to the interval maximum value, and the decision probability that the first subband output by the noise reduction model belongs to a preset category determines the gains of other subbands except the first subband.

其中，预设频点值到区间最大值的平均增益，可以认为是从预设频点值到区间最大值的频率范围内，各频点对应的增益的平均值。由于第一子带对应的频率区间中，频率值较大的频点的增益与相对的高频区间对应的其他子带的增益关联性较高。因此，可以将第一子带的频率区间中，预设频点值到区间最大值的平均增益，作为确定其他子带的增益的因素之一。Wherein, the average gain from the preset frequency point value to the interval maximum value may be considered as the average value of the gain corresponding to each frequency point within the frequency range from the preset frequency point value to the interval maximum value. Because in the frequency interval corresponding to the first subband, the gain of a frequency point with a larger frequency value is highly correlated with the gain of other subbands corresponding to the opposite high frequency interval. Therefore, in the frequency interval of the first subband, the average gain from the preset frequency point value to the interval maximum value may be used as one of the factors for determining the gains of other subbands.

其中，降噪模型除了可以对第一子带的频率信息进行降噪之外，还可以对音频帧数据进行有效声音类别的检测，其中有效声音类别例如为人声类别，或是乐器声类别等。当预设类别的判决概率的数值越大，可以认为该音频帧数据属于噪声的概率越小，相应的增益越大。由于同一音频帧数据中，各子带属于预设类别的判决概率的关联性较高。因此，可以将第一子带属于预设类别的判决概率，也作为确定其他子带的增益的因素之一。The noise reduction model can perform noise reduction on the frequency information of the first subband, and can also detect effective sound types on the audio frame data. When the value of the judgment probability of the preset category is larger, it can be considered that the probability that the audio frame data belongs to noise is smaller, and the corresponding gain is larger. Because in the same audio frame data, the correlation of the decision probabilities that each subband belongs to the preset category is relatively high. Therefore, the decision probability that the first subband belongs to a preset category may also be used as one of the factors for determining the gains of other subbands.

在这些可选的实施方式中，通过综合第一子带的频率区间中，预设频点值到区间最大值的平均增益，以及降噪模型输出的第一子带属于预设类别的判决概率，能够确定降噪效果较好的其他子带的增益。In these optional embodiments, the average gain from the preset frequency point value to the maximum value of the interval in the frequency interval of the first subband, and the decision probability that the first subband output by the noise reduction model belongs to the preset category is obtained by synthesizing the frequency interval. , the gain of other subbands with better noise reduction effect can be determined.

S140、根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。S140. Determine the audio frame data after noise reduction according to the frequency domain information after noise reduction of the first subband and the result of performing noise reduction processing on other subbands.

本公开实施例中，可根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，得到降噪后的音频帧数据的时域信息。此外，还可对各降噪后的音频帧数据进行合成，以还原成降噪后的音频数据流或音频数据文件，以便于对时域的音频数据流或音频文件进行播放。In the embodiment of the present disclosure, the time domain information of the denoised audio frame data may be obtained according to the denoised frequency domain information of the first subband and the result of performing denoising processing on other subbands. In addition, each denoised audio frame data can also be synthesized to restore the denoised audio data stream or audio data file, so as to facilitate the playback of the time domain audio data stream or audio file.

在本公开实施例一些可选的实现方式中，音频数据的降噪方法应用于语音数据降噪，相应的，区间最大值最小的频率区间，包含人声频率区间。In some optional implementations of the embodiments of the present disclosure, the method for noise reduction of audio data is applied to noise reduction of speech data, and correspondingly, the frequency interval with the smallest interval maximum value includes the human voice frequency interval.

在这些可选的实施方式中，降噪方法可应用于语音数据降噪，例如可在噪声较大的通讯环境下，对通讯者输入的语音流进行实时降噪，或者对已录制的语音文件进行降噪等等。并且，在应用于语音数据降噪时，区间最大值最小的频率区间，即语音数据中相对低频的区间需要包含人声频率的区间，以提高语音数据的降噪效果。In these optional implementations, the noise reduction method can be applied to voice data noise reduction. For example, in a communication environment with high noise, real-time noise reduction can be performed on the voice stream input by the communicator, or the recorded voice file can be denoised in real time. Noise reduction, etc. Moreover, when applied to voice data noise reduction, the frequency zone with the smallest interval maximum value, that is, the relatively low frequency zone in the voice data, needs to include the voice frequency zone, so as to improve the noise reduction effect of the voice data.

本公开实施例的技术方案，把分帧后的音频数据划分为各个子带，通过降噪模型只对低频区间对应的第一子带进行降噪，并根据降噪模型输出的第一子带的增益，对其他子带的增益进行映射，以对其他子带进行降噪。相较于传统的利用降噪模型对全频带降噪来说，减少了整体的降噪耗时，提高了降噪效率。In the technical solution of the embodiment of the present disclosure, the divided audio data is divided into sub-bands, only the first sub-band corresponding to the low-frequency interval is de-noised by the noise-reduction model, and the first sub-band outputted by the noise-reduction model is used for noise reduction. , and map the gains of other subbands to denoise the other subbands. Compared with the traditional noise reduction model for full-band noise reduction, the overall noise reduction time is reduced and the noise reduction efficiency is improved.

此外，由于降噪模型只应用于第一子带降噪，而非全频带降噪，降噪模型可以只通第一子带的样本数据进行训练，从而能够提高模型训练效率。尤其针对高采样率的音频数据进行模型训练的情况，通过只对第一子带的样本数据进行训练，可大大减少模型参数和模型复杂度，提高模型训练效率。In addition, since the noise reduction model is only applied to the first sub-band noise reduction, not the full-band noise reduction, the noise reduction model can be trained only through the sample data of the first sub-band, thereby improving the model training efficiency. Especially in the case of model training with high sampling rate audio data, by training only the sample data of the first subband, model parameters and model complexity can be greatly reduced, and model training efficiency can be improved.

实施例二Embodiment 2

本公开实施例与上述实施例中所提供的音频数据的降噪方法中各个可选方案可以结合。本实施例所提供的音频数据的降噪方法，对在时域上进行分子带处理，以及相应的在时域合成降噪后音频帧数据等步骤进行了优化，丰富了分子带处理方式。并且，通过进行下采样处理，能够减少降噪模型的计算量，在一定程度上提高降噪效率，通过进行上采样处理，够保证音频帧数据的音质。The embodiments of the present disclosure may be combined with various optional solutions in the noise reduction method for audio data provided in the above-mentioned embodiments. The noise reduction method for audio data provided in this embodiment optimizes the steps of performing molecular band processing in the time domain, and correspondingly synthesizing audio frame data after noise reduction in the time domain, thereby enriching the molecular band processing methods. In addition, by performing down-sampling processing, the calculation amount of the noise reduction model can be reduced, and the noise reduction efficiency can be improved to a certain extent. By performing up-sampling processing, the sound quality of the audio frame data can be guaranteed.

图2为本公开实施例二所提供的一种音频数据的降噪方法的流程示意图。参见图2，本实施例提供的音频数据的降噪方法，包括：FIG. 2 is a schematic flowchart of a noise reduction method for audio data according to Embodiment 2 of the present disclosure. Referring to FIG. 2, the noise reduction method for audio data provided by this embodiment includes:

S210、在时域对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带。S210. Perform molecular band processing on the audio frame data in the time domain, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband.

其中，在时域对音频帧数据进行分子带处理，可以包括：通过分析滤波器组对音频帧数据进行分子带处理，其中分析滤波器组中各滤波器的通带不同。Wherein, performing molecular band processing on the audio frame data in the time domain may include: performing molecular band processing on the audio frame data through an analysis filter bank, wherein each filter in the analysis filter bank has different passbands.

分析滤波器组中的滤波器可以是正交镜像滤波器，也可以是离散余弦调制的滤波器，或者可以是其他可应用于分子带处理的滤波器，在此不做穷举。其中，滤波器组中滤波器的数量可以等于用户需求的子带的数量，且各滤波器的通带，可以与各子带需求的频率区间相对应，例如可以分别等于各子带需求的频率区间。通过将时域的音频帧数据与各滤波器分别进行卷积，可以得到不同子带的时域信息，以实现不同子带的划分。The filters in the analysis filter bank can be quadrature mirror filters, discrete cosine modulated filters, or other filters that can be applied to molecular band processing, which are not exhaustive here. The number of filters in the filter bank can be equal to the number of subbands required by the user, and the passband of each filter can correspond to the frequency range required by each subband, for example, can be respectively equal to the required frequency of each subband interval. By convolving the audio frame data in the time domain with each filter respectively, time domain information of different subbands can be obtained, so as to realize the division of different subbands.

S220、将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益。S220. Input the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband and the gain of the first subband.

本公开实施例中，在得到各子带的时域信息后，可以只将第一子带的时域信息进行时频变换处理，而不用对其他子带的时域数据进行变换。其中，可以将第一子带的时域信息进行傅里叶变换，以得到第一子带的频域信息。通过利用降噪模型，能够实现第一子带的频域信息降噪，以及输出第一子带对应的频域区间中，各频点对应的增益。In the embodiment of the present disclosure, after obtaining the time domain information of each subband, only the time domain information of the first subband may be subjected to time-frequency transform processing, without transforming the time domain data of other subbands. Wherein, the time domain information of the first subband may be Fourier transformed to obtain the frequency domain information of the first subband. By using the noise reduction model, the noise reduction of the frequency domain information of the first subband can be realized, and the gain corresponding to each frequency point in the frequency domain interval corresponding to the first subband can be output.

S230、基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带的时域信息进行降噪处理。S230. Determine the gains of other subbands except the first subband based on the gain of the first subband, and perform noise reduction processing on the time domain information of the other subbands according to the gains of the other subbands.

其中，在得到其他子带的增益之后，可以直接利用其他子带的时域信息乘以相对应的增益，以实现对其他子带的时域信息进行降噪处理。Wherein, after the gain of the other subbands is obtained, the time domain information of the other subbands may be directly multiplied by the corresponding gain to implement noise reduction processing on the time domain information of the other subbands.

S240、将第一子带降噪后的频域信息转化为时域信息，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成，得到降噪后的音频帧数据的时域信息。S240. Convert the frequency domain information after the noise reduction of the first subband into time domain information, and synthesize the time domain information after the noise reduction of the first subband and the time domain information after the noise reduction of other subbands, to obtain the after noise reduction The time domain information of the audio frame data.

其中，可以利用傅里叶逆变换，将第一子带降噪后的频域信息转化为时域信息。通过将各子带降噪后的时域信息合成，可以得到降噪后的音频帧数据的时域信息。The inverse Fourier transform can be used to convert the frequency domain information after noise reduction of the first subband into time domain information. By synthesizing the noise-reduced time-domain information of each subband, the time-domain information of the noise-reduced audio frame data can be obtained.

其中，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成，可以包括：通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成。其中，综合滤波器组中的各滤波器，需要与分析滤波器组中的各滤波器相对应，以实现将分子带后的音频帧数据进行准确的合成。Wherein, synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands may include: synthesizing the denoised time domain information of the first subband and the denoised time domain information of the first subband through a synthetic filter bank. The denoised temporal information of other subbands is synthesized. Among them, each filter in the synthesis filter bank needs to correspond to each filter in the analysis filter bank, so as to realize accurate synthesis of the audio frame data after the molecular band.

在本公开实施例一些进一步的实现方式中，还包括：在通过分析滤波器组对音频帧数据进行分子带处理之前，对音频帧数据进行下采样处理；或者，在通过分析滤波器组对音频帧数据进行分子带处理之后，对第一子带的时域信息进行下采样处理；In some further implementations of the embodiments of the present disclosure, the method further includes: before performing molecular band processing on the audio frame data by using the analysis filter bank, performing downsampling processing on the audio frame data; After the frame data is subjected to molecular band processing, down-sampling processing is performed on the time domain information of the first subband;

相应的，在通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之前，对第一子带降噪后的时域信息和其他子带降噪后的时域信息上采样处理；或者，在通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之后，对降噪后的音频帧数据的时域信息进行上采样处理。Correspondingly, before synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands through the synthesis filter bank, the denoised time domain information and Up-sampling processing of the time-domain information after noise reduction of other subbands; or, after synthesizing the time-domain information after noise reduction of the first subband and the time-domain information after noise reduction of other subbands through a synthesis filter bank, Up-sampling processing is performed on the time domain information of the denoised audio frame data.

其中，根据Nobel等式可知，先对音频帧数据进行下采样处理，再通过分析滤波器组对音频帧数据进行分子带处理，可以等效于，先通过分析滤波器组对音频帧数据进行分子带处理，再对第一子带的时域信息进行下采样处理。同理，先对第一子带降噪后的时域信息和其他子带降噪后的时域信息上采样处理，再通过综合滤波器组，对上采样后的时域信息进行合成，可以等效于，先通过综合滤波器组，将各子带噪后的时域信息进行合成之后，再对合成的时域信息进行上采样处理。Among them, according to the Nobel equation, it can be known that the audio frame data is first subjected to downsampling processing, and then the numerator band processing is performed on the audio frame data through the analysis filter bank. Band processing, and then perform down-sampling processing on the time domain information of the first subband. Similarly, first up-sampling the time-domain information after noise reduction in the first subband and the time-domain information after noise reduction in other subbands, and then synthesizing the up-sampled time-domain information through a comprehensive filter bank, you can It is equivalent to firstly synthesizing the noised time-domain information of each subband through a synthesis filter bank, and then performing up-sampling processing on the synthesized time-domain information.

在这些进一步的实现方式中，通过下采样的操作，能够实现减少降噪模型待计算的数据量，从而可以在一定程度上提高降噪模型的滤波效率。通过上采样的操作，能够在一定程度上还原音频帧数据的音质，保证播放音频帧数据时用户的听觉体验。In these further implementation manners, the down-sampling operation can reduce the amount of data to be calculated by the noise reduction model, so that the filtering efficiency of the noise reduction model can be improved to a certain extent. Through the upsampling operation, the sound quality of the audio frame data can be restored to a certain extent, so as to ensure the user's listening experience when the audio frame data is played.

示例性的，当用户需求的子带数量为3个时，对音频帧数据x(n),n＝1,2,...L(其中L为样本点总数)进行降噪的过程，可以是：Exemplarily, when the number of subbands required by the user is 3, the process of performing noise reduction on the audio frame data x(n), n=1, 2, . . . L (where L is the total number of sample points) can be Yes:

首先，可以针对子带数量，以及第一子带的频率范围进行滤波器组的设计。First, the filter bank can be designed according to the number of subbands and the frequency range of the first subband.

例如，可采用3个离散余弦调制的滤波器，对该音频帧数据进行分子带处理，且各滤波器的表达式可以为：For example, three discrete cosine modulated filters can be used to perform molecular band processing on the audio frame data, and the expressions of each filter can be:

其中，h₀(n)可以表示原型低通滤波器，h₁(n)、h₂(n)和h₃(n)可分别表示3个离散余弦调制的滤波器；其中，N可以等于滤波器的数量，滤波器长度可为len。Among them, h₀ (n) can represent a prototype low-pass filter, and h₁ (n), h₂ (n) and h₃ (n) can respectively represent three discrete cosine modulated filters; where N can be equal to the filter The number of filters, the filter length can be len.

同时，可以对音频帧数据进行下采样。At the same time, the audio frame data may be down-sampled.

例如，可以将音频帧数据进行M倍的下采样处理，且下采样处理的表达式可以为：For example, the audio frame data can be down-sampled by M times, and the expression of down-sampling can be:

其中，dec(m,n)可以表示下次样后第m组的第n个采样点的时域信息；M可以等于子带的个数。Wherein, dec(m,n) may represent the time domain information of the nth sampling point of the mth group after the next sample; M may be equal to the number of subbands.

其次，可以利用滤波器h₁(n)、h₂(n)和h₃(n)分别对各组下采样处理后的音频帧数据进行分子带处理，且分子带处理的表达式可以为：Secondly, filters h₁ (n), h₂ (n) and h₃ (n) can be used to perform molecular band processing on the audio frame data after each group of down-sampling processing respectively, and the expression of molecular band processing can be as follows:

x(m,n)＝dec(m,n)×h_m(n)；x(m,n)=dec(m,n)×_hm (n);

其中，x(m,n)可以表示第m个子带的第n个点的时域信息。通过将每组下采样的时域信息，分别与分析滤波器组中的各滤波器进行时域卷积，能够得到各子带的时域信息。Wherein, x(m,n) may represent the time domain information of the nth point of the mth subband. The time domain information of each subband can be obtained by convolving the time domain information of each group with each filter in the analysis filter bank respectively.

再次，可以只对第一子带的时域信息做

个点的快速傅里叶变换(fast Fouriertransform，FFT)，其它子带不用作此处理。且第一个子带的时域信息的FFT变换，可以简化表示为：Again, it is possible to do only the time domain information of the first subband

Fast Fourier transform (FFT) of each point, other subbands are not used for this processing. And the FFT transform of the time domain information of the first subband can be simplified as:

其中，S(1,k)可以表示第一子带中第k个点对应的频域信息；其中，x(1,n)可以表示第1个子带(即第一子带)的第n个点的时域信息；其中，

可以表示对第一子带进行FFT变换。Wherein, S(1,k) may represent the frequency domain information corresponding to the kth point in the first subband; where x(1,n) may represent the nth point of the first subband (ie, the first subband) time domain information of the point; where,

It can be expressed that the FFT transform is performed on the first subband.

接着，将S(1,k)输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带的时域信息进行降噪处理；将第一子带降噪后的频域信息转化为时域信息。Next, input S(1,k) into the noise reduction model, so that the noise reduction model outputs the frequency domain information after the noise reduction of the first subband and the gain of the first subband; Gains of other subbands outside the first subband, and perform noise reduction processing on the time domain information of other subbands according to the gains of other subbands; convert the frequency domain information after noise reduction of the first subband into time domain information .

然后，通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成。Then, by synthesizing the filter bank, the denoised time domain information of the first subband and the denoised time domain information of other subbands are synthesized.

其中，综合滤波器组中的各滤波器，需要与分析滤波器组中的各滤波器相对应。例如，可采用3个离散余弦调制的滤波器，作为综合滤波器组中的各滤波器，且综合滤波器组中各滤波器的表达式可以为：Among them, each filter in the synthesis filter bank needs to correspond to each filter in the analysis filter bank. For example, three discrete cosine modulated filters can be used as each filter in the synthesis filter bank, and the expression of each filter in the synthesis filter bank can be:

其中，综合滤波器组中滤波器f₁(n)、f₂(n)和f₃(n)，与h₁(n)、h₂(n)和h₃(n)分别对应。通过f₁(n)、f₂(n)和f₃(n)可将各子带降噪后的时域信息进行滤波，且滤波的表达式可以为：Among them, the filters f₁ (n), f₂ (n) and f₃ (n) in the synthesis filter bank correspond to h₁ (n), h₂ (n) and h₃ (n) respectively. Through f₁ (n), f₂ (n) and f₃ (n), the time-domain information after noise reduction of each subband can be filtered, and the filtering expression can be:

syn(m,n)＝out(m,n)×f_m(n)；syn(m,n)=out(m,n)×_fm (n);

其中，out(m,n)可以表示降噪后第m个子带的第n个采样点的时域信息；syn(m,n)可以表示第m个子带第n个采用点滤波后的时域信息。通过将各个子带的时域信息与对应的子带合成滤波器作卷积，能够为降噪后音频帧数据的合成奠定基础。Among them, out(m,n) can represent the time domain information of the nth sampling point of the mth subband after noise reduction; syn(m,n) can represent the time domain information of the mth subband after point filtering. information. By convolving the time domain information of each subband with the corresponding subband synthesis filter, a foundation can be laid for the synthesis of the audio frame data after noise reduction.

最后，将syn(m,n)进行M倍的上采样处理up(m,n)＝upsample(syn(m,n))，并综合得到最终降噪增强后的音频帧数据，且综合的表达式可以为：Finally, syn(m,n) is subjected to M times of upsampling processing up(m,n)=upsample(syn(m,n)), and the final audio frame data after noise reduction and enhancement is obtained comprehensively, and the comprehensive expression The formula can be:

其中，enh(n)可表示最终的增强音频数据。Among them, enh(n) may represent the final enhanced audio data.

本公开实施例的技术方案，对在时域上进行分子带处理，以及相应的在时域合成降噪后音频帧数据等步骤进行了优化，丰富了分子带处理方式。并且，通过进行下采样处理，能够减少降噪模型的计算量，在一定程度上提高降噪效率，通过进行上采样处理，够保证音频帧数据的音质。此外，本公开实施例提供的音频数据的降噪方法与上述实施例提供的音频数据的降噪方法属于同一技术构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且相同的技术特征在本实施例与上述实施例中具有相同的有益效果。The technical solutions of the embodiments of the present disclosure optimize the steps of performing molecular band processing in the time domain, and correspondingly synthesizing audio frame data after noise reduction in the time domain, and enriching the molecular band processing methods. In addition, by performing down-sampling processing, the calculation amount of the noise reduction model can be reduced, and the noise reduction efficiency can be improved to a certain extent. By performing up-sampling processing, the sound quality of the audio frame data can be guaranteed. In addition, the noise reduction method for audio data provided by the embodiment of the present disclosure and the noise reduction method for audio data provided by the above-mentioned embodiments belong to the same technical concept. The technical features of this embodiment have the same beneficial effects as the above-mentioned embodiments.

实施例三Embodiment 3

本公开实施例与上述实施例中所提供的音频数据的降噪方法中各个可选方案可以结合。本实施例所提供的音频数据的降噪方法，对在频域上进行分子带处理，以及相应的在频域合成降噪后音频帧数据等步骤进行了优化，丰富了分子带处理方式。The embodiments of the present disclosure may be combined with various optional solutions in the noise reduction method for audio data provided in the above-mentioned embodiments. The noise reduction method for audio data provided in this embodiment optimizes the steps of performing molecular band processing in the frequency domain, and correspondingly synthesizing audio frame data after noise reduction in the frequency domain, thereby enriching the molecular band processing methods.

图3为本公开实施例三所提供的一种音频数据的降噪方法的流程示意图。参见图3，本实施例提供的音频数据的降噪方法，包括：FIG. 3 is a schematic flowchart of a noise reduction method for audio data according to Embodiment 3 of the present disclosure. Referring to FIG. 3, the noise reduction method for audio data provided by this embodiment includes:

S310、将音频帧数据转化为频域信息后，对音频帧数据的频域信息进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带。S310. After converting the audio frame data into frequency domain information, perform molecular band processing on the frequency domain information of the audio frame data, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as first subband.

其中，可以将音频帧数据的时域信息直接进行傅里叶变换，以得到全频带音频帧数据的频域信息。进而，对音频帧数据的频域信息进行分子带处理，可以包括：将音频帧数据的频域信息按频率区间进行分组，得到各子带的频域信息。The time domain information of the audio frame data can be directly Fourier transformed to obtain the frequency domain information of the full-band audio frame data. Furthermore, performing molecular band processing on the frequency domain information of the audio frame data may include: grouping the frequency domain information of the audio frame data into frequency intervals to obtain the frequency domain information of each subband.

其中，音频帧数据的频域信息，可以包括当前音频帧数据进行时频转化后，各采样点对应的每个频点的分量信息，例如各采样点在1khz、2khz、3khz、...24khz等频点处各包含多少分量。将音频帧数据的频域信息按频率区间分组，可以理解为按频点划分频率区间，并将各频率区间内的频域信息作为不同子带的频域信息。The frequency domain information of the audio frame data may include the component information of each frequency point corresponding to each sampling point after the time-frequency conversion of the current audio frame data, for example, each sampling point is at 1khz, 2khz, 3khz, ... 24khz How many components are included at each frequency point. Grouping the frequency domain information of the audio frame data by frequency intervals can be understood as dividing the frequency intervals by frequency points, and using the frequency domain information in each frequency interval as the frequency domain information of different subbands.

示例性的，按频点划分频率区间，例如是若全频带的频率范围为0hz-24khz，则可以将8khz和16khz两个频点作为划分频点，将0hz-24khz划分为0hz-8khz、8khz-16khz和16khz-24khz三个频率区间。各频率区间内的频域信息，可以认为是不同子带的频域信息。Exemplarily, the frequency interval is divided according to frequency points, for example, if the frequency range of the whole frequency band is 0hz-24khz, two frequency points of 8khz and 16khz can be used as dividing frequency points, and 0hz-24khz is divided into 0hz-8khz, 8khz -16khz and 16khz-24khz three frequency intervals. The frequency domain information in each frequency interval can be regarded as the frequency domain information of different subbands.

S320、将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益。S320. Input the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband and the gain of the first subband.

其中，在确定第一子带的频域信息之后，可以只将第一子带的频域信息输入降噪模型，而不用将其他子带的频域数据输入降噪模型。Wherein, after the frequency domain information of the first subband is determined, only the frequency domain information of the first subband may be input into the noise reduction model, without inputting the frequency domain data of other subbands into the noise reduction model.

S330、基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带的频域信息进行降噪处理。S330. Determine the gains of other subbands except the first subband based on the gain of the first subband, and perform noise reduction processing on the frequency domain information of the other subbands according to the gains of the other subbands.

其中，在得到其他子带的增益之后，可以直接利用其他子带的频域信息乘以相对应的增益，以实现对其他子带的频域信息进行降噪处理。Wherein, after the gain of the other subbands is obtained, the frequency domain information of the other subbands may be directly multiplied by the corresponding gain to implement noise reduction processing on the frequency domain information of the other subbands.

S340、将第一子带降噪后的频域信息，以及其他子带降噪后的频域信息进行拼接，得到音频帧数据降噪后的频域信息。S340 , splicing the denoised frequency domain information of the first subband and the denoised frequency domain information of other subbands to obtain denoised frequency domain information of the audio frame data.

其中，将第一子带降噪后的频域信息，以及其他子带降噪后的频域信息进行拼接，包括：将第一子带降噪后的频域信息，以及其他子带降噪后的频域信息按频率区间进行拼接。Wherein, the frequency domain information after noise reduction of the first subband and the frequency domain information after noise reduction of other subbands are spliced, including: the frequency domain information after noise reduction of the first subband and the noise reduction of other subbands are combined. The latter frequency domain information is spliced by frequency interval.

各频域信息按频域区间进行了分子带，通过对降噪后各子带的频域信息，按照划分的频域区间再次进行拼接，能够得到音频帧数据降噪后的全频带的频域信息。Each frequency domain information is divided into molecular bands according to the frequency domain interval. By splicing the frequency domain information of each subband after noise reduction according to the divided frequency domain intervals, the frequency domain of the whole frequency band after noise reduction of the audio frame data can be obtained. information.

S350、将音频帧数据降噪后的频域信息转化为时域信息，得到降噪后音频数据的帧数据的时域信息。S350. Convert the frequency domain information after noise reduction of the audio frame data into time domain information to obtain time domain information of the frame data of the denoised audio data.

其中，可以利用傅里叶逆变换，将音频帧数据降噪后的频域信息转化为时域信息，即可以得到降噪后的音频帧数据的时域信息。The inverse Fourier transform can be used to convert the frequency domain information of the denoised audio frame data into time domain information, that is, the time domain information of the denoised audio frame data can be obtained.

示例性的，当用户需求的子带数量为3个时，对音频帧数据x(n),n＝1,2,...L(其中L为样本点总数)进行降噪的过程，还可以是：Exemplarily, when the number of subbands required by the user is 3, the process of noise reduction is performed on the audio frame data x(n), n=1, 2, . . . L (where L is the total number of sample points), and further Can be:

首先，对音频帧数据x(n)做L个点的FFT变换，可简化表示为：First, perform L-point FFT transformation on the audio frame data x(n), which can be simplified as:

其中，X(k)为音频帧数据中第k个点的频域信息。Among them, X(k) is the frequency domain information of the kth point in the audio frame data.

其次，将X(k)按频率区间进行分组，得到各子带的频域信息，可简化表示为：Secondly, group X(k) according to the frequency interval to obtain the frequency domain information of each subband, which can be simplified as:

其中，M可以表示子带的个数；S(m,k)可以表示第m子带中第k个点对应的频域信息。Wherein, M may represent the number of subbands; S(m, k) may represent the frequency domain information corresponding to the kth point in the mth subband.

再次，可以将将S(1,k)输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带的频域信息进行降噪处理。Again, S(1,k) can be input into the noise reduction model, so that the noise reduction model outputs the frequency domain information after the noise reduction of the first subband, and the gain of the first subband; based on the gain of the first subband, Gains of other subbands except the first subband are determined, and noise reduction processing is performed on the frequency domain information of the other subbands according to the gains of the other subbands.

接着，可以将所有子带S(m,k)拼起来，得到全带频域信息S(k)，通过FFT逆变换，可以得到降噪后音频数据的帧数据的时域信息，且FFT逆变换可简化表示为：Next, all the subbands S(m,k) can be spliced together to obtain the full-band frequency domain information S(k). Through the inverse FFT transformation, the time domain information of the frame data of the denoised audio data can be obtained, and the FFT inverse The transformation can be simplified as:

enh(n)＝ifft(S(k))；enh(n)=ifft(S(k));

其中，enh(n)可表示最终的增强音频数据。Among them, enh(n) may represent the final enhanced audio data.

本公开实施例的技术方案，对在频域上进行分子带处理，以及相应的在频域合成降噪后音频帧数据等步骤进行了优化，丰富了分子带处理方式。此外，本公开实施例提供的音频数据的降噪方法与上述实施例提供的音频数据的降噪方法属于同一技术构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且相同的技术特征在本实施例与上述实施例中具有相同的有益效果。The technical solutions of the embodiments of the present disclosure optimize the steps of performing molecular band processing in the frequency domain, and correspondingly synthesizing audio frame data after noise reduction in the frequency domain, and enriching the molecular band processing methods. In addition, the noise reduction method for audio data provided by the embodiment of the present disclosure and the noise reduction method for audio data provided by the above-mentioned embodiments belong to the same technical concept. The technical features of this embodiment have the same beneficial effects as the above-mentioned embodiments.

实施例四Embodiment 4

本公开实施例与上述实施例中所提供的音频数据的降噪方法中各个可选方案可以结合。本实施例所提供的音频数据的降噪方法，对确定其他子带的增益步骤进行了优化，能够实现从第一子带的增益向其他子带的增益的映射，为其他子带的降噪处理奠定基础。The embodiments of the present disclosure may be combined with various optional solutions in the noise reduction method for audio data provided in the above-mentioned embodiments. The method for noise reduction of audio data provided in this embodiment optimizes the step of determining the gain of other subbands, and can realize the mapping from the gain of the first subband to the gain of other subbands, so that the noise reduction of other subbands can be realized. Processing lays the foundation.

在本公开实施例一些可选的实现方式中，综合第一子带的频率区间中，预设频点值到区间最大值的平均增益，以及降噪模型输出的第一子带属于预设类别的判决概率，确定除第一子带外的其他子带的增益，包括：In some optional implementations of the embodiments of the present disclosure, in the frequency range of the integrated first subband, the average gain from the preset frequency point value to the maximum value of the range, and the first subband output by the noise reduction model belong to a preset category The decision probability of , determines the gain of other subbands except the first subband, including:

将第一子带的频率区间中，预设频点值到区间最大值的平均增益，作为第一增益因子；根据第一增益因子，以及降噪模型输出的第一子带属于预设类别的判决概率，确定除第一子带外的其他子带属于预设类别的判决概率；根据其他子带属于预设类别的判决概率，确定其他子带的第二增益因子；综合第一增益因子、第二增益因子，以及其他子带属于预设类别的判决概率，确定除第一子带外的其他子带的增益。In the frequency interval of the first subband, the average gain from the preset frequency point value to the maximum value of the interval is taken as the first gain factor; according to the first gain factor, and the first subband output by the noise reduction model belongs to the preset category. decision probability, determine the decision probability that other subbands except the first subband belong to the preset category; determine the second gain factor of other subbands according to the decision probability that other subbands belong to the preset category; integrate the first gain factor, The second gain factor, and the decision probabilities that other subbands belong to a preset category, determine the gains of other subbands except the first subband.

示例性的，假设当前音频帧数据中，第m个子带第k个频的增益记为G(m,k)，且为方便描述，第一子带的增益G(1,k)可以简化表示为g(k)。Exemplarily, it is assumed that in the current audio frame data, the gain of the kth frequency of the mth subband is denoted as G(m,k), and for the convenience of description, the gain G(1,k) of the first subband can be simplified to represent is g(k).

由于第一子带对应的频率区间中，频率值较大的频点的增益与相对的高频区间对应的其他子带的增益关联性较高。因此，可以将第一子带的频率区间中，预设频点值到区间最大值的平均增益，作为确定其他子带的增益的第一增益因子，以保证根据增益降噪后的子带之间可以平滑过渡。Because in the frequency interval corresponding to the first subband, the gain of a frequency point with a larger frequency value is highly correlated with the gain of other subbands corresponding to the opposite high frequency interval. Therefore, in the frequency interval of the first subband, the average gain from the preset frequency point value to the maximum value of the interval can be used as the first gain factor for determining the gains of other subbands, so as to ensure that the subbands after noise reduction according to the gain There is a smooth transition between them.

其中，第一增益因子的计算公式，可以简化表示为：Among them, the calculation formula of the first gain factor can be simplified as:

avgGainH可表示第一增益因子；nFFT为常数值，可表示时频变换后频点的总数量；Bw也为常数值，可表示预设频点值到区间最大值之间的频点数量，一般取经验值，例如可以是nFFT的1/4到1/3范围内的数值。avgGainH can represent the first gain factor; nFFT is a constant value, which can represent the total number of frequency points after time-frequency transformation; Bw is also a constant value, which can represent the number of frequency points between the preset frequency point value and the interval maximum value, generally Take an empirical value, for example, it can be a value in the range of 1/4 to 1/3 of nFFT.

其中，降噪模型估计出的第一子带属于预设类别的判决概率，可以认为是当前音频帧数据属于预设类别的判决概率，且可记作vad。The decision probability that the first subband estimated by the noise reduction model belongs to the preset category may be considered as the judgment probability that the current audio frame data belongs to the preset category, and may be denoted as vad.

其中，根据第一增益因子avgGainH，以及降噪模型输出的第一子带属于预设类别的判决概率vad，确定除第一子带外的其他子带属于预设类别的判决概率，例如可以基于下述公式确定：Wherein, according to the first gain factor avgGainH and the decision probability vad that the first subband output by the noise reduction model belongs to the preset category, determine the decision probability that other subbands except the first subband belong to the preset category, for example, it can be based on Determined by the following formula:

avgProbH可以表示除第一子带外的其他子带属于预设类别的判决概率。avgProbH除了通过确定avgGainH和vad的平方根来得到之外，还可以通过确定avgGainH和vad的加权和得到，且其他融合avgGainH和vad，以确定除第一子带外的其他子带属于预设类别的判决概率的方式也可应用于此，在此不做具体限定。avgProbH may represent the decision probability that other subbands except the first subband belong to a preset category. In addition to determining the square root of avgGainH and vad, avgProbH can also be obtained by determining the weighted sum of avgGainH and vad, and fuses avgGainH and vad to determine that other subbands except the first subband belong to the preset category. The method of determining probability can also be applied to this, which is not specifically limited here.

其中，根据其他子带属于预设类别的判决概率avgProbH，确定其他子带的第二增益因子，例如可以基于下述公式确定：Wherein, according to the decision probability avgProbH of the other subbands belonging to the preset category, the second gain factor of the other subbands is determined, for example, it can be determined based on the following formula:

gainH可以表示其他子带的第二增益因子，且gainH与avgProbH需呈正相关，值域范围需是[0,1]。除了通过双曲正切tanh确定gainH之外，其他可满足gainH与avgProbH呈正相关，gainH值域范围是[0,1]的gainH的确定方式也可应用于此，在此不做具体限定。gainH can represent the second gain factor of other subbands, and gainH and avgProbH must be positively correlated, and the value range must be [0,1]. In addition to determining gainH by the hyperbolic tangent tanh, other methods for determining gainH that can satisfy the positive correlation between gainH and avgProbH, and the gainH value range is [0,1] can also be applied to this, which is not specifically limited here.

其中，综合第一增益因子avgGainH、第二增益因子gainH，以及其他子带属于预设类别的判决概率avgProbH，确定除第一子带外的其他子带的增益，例如可以基于下述公式确定：Wherein, synthesizing the first gain factor avgGainH, the second gain factor gainH, and the decision probability avgProbH that other subbands belong to the preset category, to determine the gain of other subbands except the first subband, for example, it can be determined based on the following formula:

其中，gain可表示除第一子带外的其他子带的增益。由于除第一子带外的其他子带对应的相对的高频区间，包含有效声音的分量信息较少，通过将其他子带中各频点的增益，设置为相同增益数值，能够满足对其他子带的降噪需求。Wherein, gain may represent gains of other subbands except the first subband. Since the relative high frequency interval corresponding to other subbands except the first subband contains less component information of effective sound, by setting the gain of each frequency point in the other subbands to the same gain value, it can satisfy the requirements for other subbands. Noise reduction requirements for subbands.

本公开实施例的技术方案，对确定其他子带的增益步骤进行了优化，能够实现从第一子带的增益向其他子带的增益的映射，为其他子带的降噪处理奠定基础。此外，本公开实施例提供的音频数据的降噪方法与上述实施例提供的音频数据的降噪方法属于同一技术构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且相同的技术特征在本实施例与上述实施例中具有相同的有益效果。The technical solutions of the embodiments of the present disclosure optimize the step of determining the gain of other subbands, and can realize the mapping from the gain of the first subband to the gain of other subbands, laying a foundation for noise reduction processing of other subbands. In addition, the noise reduction method for audio data provided by the embodiment of the present disclosure and the noise reduction method for audio data provided by the above-mentioned embodiments belong to the same technical concept. The technical features of this embodiment have the same beneficial effects as the above-mentioned embodiments.

实施例五Embodiment 5

图4为本公开实施例五所提供的一种音频数据的降噪装置结构示意图。本实施例提供的音频数据的降噪装置适用于音频数据降噪的情形，尤其适用于高采样率的音频数据的实时降噪的情形。FIG. 4 is a schematic structural diagram of a noise reduction apparatus for audio data according to Embodiment 5 of the present disclosure. The device for noise reduction of audio data provided in this embodiment is suitable for the situation of noise reduction of audio data, and is especially suitable for the situation of real-time noise reduction of audio data with a high sampling rate.

如图4所示，音频数据的降噪装置，包括：As shown in Figure 4, the noise reduction device for audio data includes:

分子带模块410，用于对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；Themolecular band module 410 is used to perform molecular band processing on the audio frame data, wherein the frequency intervals of each sub-band are different, and the sub-band corresponding to the frequency interval with the smallest interval maximum value is used as the first sub-band;

第一降噪模块420，用于将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；The firstnoise reduction module 420 is configured to input the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the denoised frequency domain information of the first subband and the gain of the first subband;

第二降噪模块430，用于基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；The secondnoise reduction module 430 is configured to determine the gains of other subbands except the first subband based on the gain of the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands;

降噪数据确定模块440，用于根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。The noise reductiondata determination module 440 is configured to determine the audio frame data after noise reduction according to the frequency domain information after noise reduction of the first subband and the result of performing noise reduction processing on other subbands.

在本公开实施例一些可选的实现方式中，分子带模块，包括：In some optional implementations of the embodiments of the present disclosure, the molecular band module includes:

时域分子带子模块，用于在时域对音频帧数据进行分子带处理；Time-domain molecular-band sub-module, which is used to perform molecular-band processing on audio frame data in time domain;

相应的，第二降噪模块，用于根据其他子带的增益分别对其他子带的时域信息进行降噪处理；Correspondingly, the second noise reduction module is configured to perform noise reduction processing on the time domain information of the other subbands according to the gains of the other subbands;

降噪数据确定模块，用于将第一子带降噪后的频域信息转化为时域信息，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成，得到降噪后的音频帧数据的时域信息。The noise reduction data determination module is used to convert the frequency domain information after the noise reduction of the first subband into time domain information, and compare the time domain information after the noise reduction of the first subband with the time domain information after the noise reduction of other subbands. Synthesize to obtain the time domain information of the denoised audio frame data.

在本公开实施例一些进一步的实现方式中，时域分子带子模块，具体用于通过分析滤波器组对音频帧数据进行分子带处理，其中分析滤波器组中各滤波器的通带不同；In some further implementations of the embodiments of the present disclosure, the time-domain molecular-band sub-module is specifically configured to perform molecular-band processing on the audio frame data through an analysis filter bank, wherein the passbands of each filter in the analysis filter bank are different;

相应的，降噪数据确定模块，具体用于通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成。Correspondingly, the noise reduction data determination module is specifically configured to synthesize the denoised time domain information of the first subband and the denoised time domain information of other subbands through the synthesis filter bank.

在本公开实施例一些进一步的实现方式中，时域分子带子模块，还用于在通过分析滤波器组对音频帧数据进行分子带处理之前，对音频帧数据进行下采样处理；或者，在通过分析滤波器组对音频帧数据进行分子带处理之后，对第一子带的时域信息进行下采样处理；In some further implementations of the embodiments of the present disclosure, the time-domain molecular band sub-module is further configured to perform downsampling processing on the audio frame data before performing the molecular band processing on the audio frame data through the analysis filter bank; After the analysis filter bank performs molecular band processing on the audio frame data, down-sampling processing is performed on the time domain information of the first subband;

相应的，降噪数据确定模块，还用于在通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之前，对第一子带降噪后的时域信息和其他子带降噪后的时域信息上采样处理；或者，在通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之后，对降噪后的音频帧数据的时域信息进行上采样处理。Correspondingly, the noise reduction data determination module is further configured to, before synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands through the synthesis filter bank, perform a The time-domain information after noise reduction and the time-domain information after noise reduction in other subbands are up-sampling processing; or, through the synthesis filter bank, the time domain information after noise reduction in the first subband and other subband noise reduction After the synthesized time domain information is synthesized, an up-sampling process is performed on the time domain information of the denoised audio frame data.

在本公开实施例一些可选的实现方式中，分子带模块，包括：In some optional implementations of the embodiments of the present disclosure, the molecular band module includes:

频域分子带子模块，用于将音频帧数据转化为频域信息后，对音频帧数据的频域信息进行分子带处理；The frequency domain molecular band sub-module is used to perform molecular band processing on the frequency domain information of the audio frame data after converting the audio frame data into frequency domain information;

相应的，第二降噪模块，用于根据其他子带的增益分别对其他子带的频域信息进行降噪处理；Correspondingly, the second noise reduction module is used to perform noise reduction processing on the frequency domain information of other subbands according to the gains of other subbands;

降噪数据确定模块，用于将第一子带降噪后的频域信息，以及其他子带降噪后的频域信息进行拼接，得到音频帧数据降噪后的频域信息；将音频帧数据降噪后的频域信息转化为时域信息，得到降噪后音频数据的帧数据的时域信息。The noise reduction data determination module is used for splicing the frequency domain information after noise reduction of the first subband and the frequency domain information after noise reduction of other subbands to obtain the frequency domain information after noise reduction of the audio frame data; The frequency domain information after data noise reduction is converted into time domain information, and the time domain information of the frame data of the denoised audio data is obtained.

在本公开实施例一些进一步的实现方式中，频域分子带子模块，具体用于将音频帧数据的频域信息按频率区间进行分组，得到各子带的频域信息；In some further implementations of the embodiments of the present disclosure, the frequency domain molecular band sub-module is specifically configured to group the frequency domain information of the audio frame data according to frequency intervals to obtain the frequency domain information of each subband;

相应的，降噪数据确定模块，具体用于将第一子带降噪后的频域信息，以及其他子带降噪后的频域信息按频率区间进行拼接。Correspondingly, the noise reduction data determination module is specifically configured to splicing the frequency domain information after noise reduction of the first subband and the frequency domain information after noise reduction of other subbands according to frequency intervals.

在本公开实施例一些可选的实现方式中，降噪模型的训练方式，包括：获取与第一子带的频率区间相同的样本帧数据，以及样本帧数据的目标频域信息；利用样本帧数据的频域信息，以及目标频域信息，对降噪模型进行训练，直至降噪模型收敛。In some optional implementations of the embodiments of the present disclosure, the training method of the noise reduction model includes: obtaining sample frame data that is the same as the frequency interval of the first subband, and target frequency domain information of the sample frame data; using the sample frame data The frequency domain information of the data and the target frequency domain information are used to train the noise reduction model until the noise reduction model converges.

在本公开实施例一些可选的实现方式中，第二降噪模块，包括：In some optional implementations of the embodiments of the present disclosure, the second noise reduction module includes:

增益映射子模块，用于综合第一子带的频率区间中，预设频点值到区间最大值的平均增益，以及降噪模型输出的第一子带属于预设类别的判决概率，确定除第一子带外的其他子带的增益。The gain mapping submodule is used to synthesize the average gain from the preset frequency point value to the maximum value of the interval in the frequency interval of the first subband, and the judgment probability that the first subband output by the noise reduction model belongs to the preset category, and determine the division Gains for other subbands other than the first subband.

在本公开实施例一些进一步的实现方式中，增益映射子模块，具体用于将第一子带的频率区间中，预设频点值到区间最大值的平均增益，作为第一增益因子；根据第一增益因子，以及降噪模型输出的第一子带属于预设类别的判决概率，确定除第一子带外的其他子带属于预设类别的判决概率；根据其他子带属于预设类别的判决概率，确定其他子带的第二增益因子；综合第一增益因子、第二增益因子，以及其他子带属于预设类别的判决概率，确定除第一子带外的其他子带的增益。In some further implementations of the embodiments of the present disclosure, the gain mapping submodule is specifically configured to use the average gain from the preset frequency point value to the maximum value of the interval in the frequency interval of the first subband as the first gain factor; The first gain factor, and the judgment probability that the first subband output by the noise reduction model belongs to the preset category, determine the judgment probability that other subbands except the first subband belong to the preset category; according to the other subbands belonging to the preset category determine the second gain factor of other subbands; combine the first gain factor, the second gain factor, and the decision probability of other subbands belonging to a preset category to determine the gain of other subbands except the first subband .

在本公开实施例一些可选的实现方式中，各子带的频率区间依次相邻。In some optional implementation manners of the embodiments of the present disclosure, the frequency intervals of each subband are adjacent in sequence.

在本公开实施例一些可选的实现方式中，音频数据的降噪装置应用于语音数据降噪，相应的，区间最大值最小的频率区间，包含人声频率区间。In some optional implementations of the embodiments of the present disclosure, the device for noise reduction of audio data is applied to noise reduction of speech data. Correspondingly, the frequency interval with the smallest interval maximum value includes the human voice frequency interval.

本公开实施例所提供的音频数据的降噪装置，可执行本公开任意实施例所提供的音频数据的降噪方法，具备执行方法相应的功能模块和有益效果。The noise reduction device for audio data provided by the embodiment of the present disclosure can execute the noise reduction method for audio data provided by any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

值得注意的是，上述装置所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。It is worth noting that the units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only For the convenience of distinguishing from each other, it is not used to limit the protection scope of the embodiments of the present disclosure.

实施例六Embodiment 6

下面参考图5，其示出了适于用来实现本公开实施例的音频数据的降噪方法的电子设备(例如图5中的终端设备或服务器)500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。Referring next to FIG. 5 , it shows a schematic structural diagram of an electronic device (eg, a terminal device or a server in FIG. 5 ) 500 suitable for implementing the noise reduction method for audio data according to an embodiment of the present disclosure. Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in FIG. 5 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(Read-Only Memory，ROM)502中的程序或者从存储装置508加载到随机访问存储器(Random Access Memory，RAM)503中的程序而执行各种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的各种程序和数据。处理装置501、ROM 502以及RAM503通过总线504彼此相连。输入/输出(I/O)接口505也连接至总线504。As shown in FIG. 5 , theelectronic device 500 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 501, which may be stored in a read-only memory (Read-Only Memory, ROM) 502 according to a program or from astorage device 508 is a program loaded into a random access memory (RAM) 503 to execute various appropriate actions and processes. In theRAM 503, various programs and data necessary for the operation of theelectronic device 500 are also stored. Theprocessing device 501 , theROM 502 , and theRAM 503 are connected to each other through abus 504 . An input/output (I/O)interface 505 is also connected tobus 504 .

通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。Typically, the following devices may be connected to the I/O interface 505:input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration Anoutput device 507 such as a computer; astorage device 508 including, for example, a magnetic tape, a hard disk, etc.; and acommunication device 509 . Communication means 509 may allowelectronic device 500 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 5 showselectronic device 500 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的音频数据的降噪方法中限定的上述功能。In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via thecommunication device 509 , or from thestorage device 508 , or from theROM 502 . When the computer program is executed by theprocessing device 501, the above-mentioned functions defined in the noise reduction method for audio data according to the embodiment of the present disclosure are executed.

本公开实施例提供的电子设备与上述实施例提供的音频数据的降噪方法属于同一公开构思，未在本公开实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。The electronic device provided by the embodiments of the present disclosure and the noise reduction method for audio data provided by the above-mentioned embodiments belong to the same disclosed concept. For technical details not described in detail in the embodiments of the present disclosure, please refer to the above-mentioned embodiments. example has the same beneficial effect.

实施例七Embodiment 7

本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的音频数据的降噪方法。Embodiments of the present disclosure provide a computer storage medium on which a computer program is stored, and when the program is executed by a processor, implements the noise reduction method for audio data provided by the foregoing embodiments.

需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)或闪存(FLASH)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Erasable Programmable Read-Only Memory (EPROM) or flash memory (FLASH), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

在一些实施方式中，客户端、服务器可以利用诸如HTTP(Hyper Text TransferProtocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。In some embodiments, the client and the server can use any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol) to communicate, and can communicate with digital data in any form or medium Communication (eg, a communication network) interconnects. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.

上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.

上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device:

对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；将第一子带的频域信息输入降噪模型，以使降噪模型输出第一子带降噪后的频域信息，和第一子带的增益；基于第一子带的增益，确定除第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；根据第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。Perform molecular band processing on the audio frame data, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband; the frequency domain information of the first subband is input into noise reduction model, so that the noise reduction model outputs the frequency domain information after the noise reduction of the first subband, and the gain of the first subband; based on the gain of the first subband, determine the gains of other subbands except the first subband, and perform noise reduction processing on other subbands according to the gains of other subbands; according to the frequency domain information after noise reduction of the first subband and the result of noise reduction processing on other subbands, determine the audio frame data after noise reduction .

可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for performing operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and This includes conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元、模块的名称在某种情况下并不构成对该单元、模块本身的限定，例如，第一降噪模块还可以被描述为“第一子带降噪模块”。The units involved in the embodiments of the present disclosure may be implemented in a software manner, and may also be implemented in a hardware manner. Wherein, the names of the units and modules do not constitute limitations on the units or modules themselves under certain circumstances. For example, the first noise reduction module may also be described as a "first subband noise reduction module".

本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FieldProgrammable Gate Array，FPGA)、专用集成电路(Application Specific IntegratedCircuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(CPLD)等等。The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts , ASSP), system on chip (System on Chip, SOC), complex programmable logic device (CPLD) and so on.

在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

根据本公开的一个或多个实施例，【示例一】提供了一种音频数据的降噪方法，该方法包括：According to one or more embodiments of the present disclosure, [Example 1] provides a noise reduction method for audio data, the method comprising:

对音频帧数据进行分子带处理，其中各子带的频率区间不同，且将区间最大值最小的频率区间对应的子带，作为第一子带；The audio frame data is subjected to molecular band processing, wherein the frequency intervals of each subband are different, and the subband corresponding to the frequency interval with the smallest interval maximum value is used as the first subband;

将所述第一子带的频域信息输入降噪模型，以使所述降噪模型输出所述第一子带降噪后的频域信息，和所述第一子带的增益；inputting the frequency domain information of the first subband into the noise reduction model, so that the noise reduction model outputs the frequency domain information after the noise reduction of the first subband and the gain of the first subband;

基于所述第一子带的增益，确定除所述第一子带外的其他子带的增益，并根据其他子带的增益分别对其他子带进行降噪处理；Based on the gain of the first subband, determine the gains of other subbands except the first subband, and perform noise reduction processing on the other subbands according to the gains of the other subbands;

根据所述第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据。The denoised audio frame data is determined according to the denoised frequency domain information of the first subband and the result of performing denoising processing on other subbands.

根据本公开的一个或多个实施例，【示例二】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 2] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述对音频帧数据进行分子带处理，包括：In some optional implementation manners of the embodiments of the present disclosure, the performing molecular band processing on the audio frame data includes:

在时域对音频帧数据进行分子带处理；Molecular band processing of audio frame data in the time domain;

相应的，所述根据其他子带的增益分别对其他子带进行降噪处理，包括：根据其他子带的增益分别对其他子带的时域信息进行降噪处理；Correspondingly, the performing noise reduction processing on the other subbands according to the gains of the other subbands includes: performing noise reduction processing on the time domain information of the other subbands respectively according to the gains of the other subbands;

所述根据所述第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据，包括：According to the frequency domain information after noise reduction of the first subband and the result of performing noise reduction processing on other subbands, the audio frame data after noise reduction is determined, including:

将所述第一子带降噪后的频域信息转化为时域信息，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成，得到降噪后的音频帧数据的时域信息。Converting the denoised frequency domain information of the first subband into time domain information, synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands to obtain a denoised The time domain information of the audio frame data.

根据本公开的一个或多个实施例，【示例三】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 3] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述在时域对音频帧数据进行分子带处理，包括：In some optional implementation manners of the embodiments of the present disclosure, the performing molecular band processing on the audio frame data in the time domain includes:

通过分析滤波器组对音频帧数据进行分子带处理，其中所述分析滤波器组中各滤波器的通带不同；Molecular band processing is performed on the audio frame data by an analysis filter bank, wherein the passbands of the filters in the analysis filter bank are different;

相应的，所述将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成，包括：通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成。Correspondingly, synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands includes: synthesizing the denoised time domain information of the first subband by synthesizing the filter bank. The information is synthesized with the temporal information after noise reduction of other subbands.

根据本公开的一个或多个实施例，【示例四】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 4] provides a noise reduction method for audio data, further comprising:

在所述通过分析滤波器组对音频帧数据进行分子带处理之前，对所述音频帧数据进行下采样处理；或者，在所述通过分析滤波器组对音频帧数据进行分子带处理之后，对第一子带的时域信息进行下采样处理；Before performing the numerator band processing on the audio frame data by analyzing the filter bank, perform down-sampling processing on the audio frame data; or, after performing the numerator band processing on the audio frame data by down-sampling the time domain information of the first subband;

相应的，在所述通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之前，对所述第一子带降噪后的时域信息和其他子带降噪后的时域信息上采样处理；或者，在所述通过综合滤波器组，将第一子带降噪后的时域信息和其他子带降噪后的时域信息进行合成之后，对降噪后的音频帧数据的时域信息进行上采样处理。Correspondingly, before synthesizing the denoised time domain information of the first subband and the denoised time domain information of other subbands through the synthesis filter bank, the denoised temporal information of the first subband is The time-domain information and the time-domain information after noise reduction of other subbands are up-sampling processing; or, in the synthesis filter bank, the time-domain information after noise reduction of the first subband and the time-domain information after noise reduction of other subbands are subjected to up-sampling processing; After the domain information is synthesized, an up-sampling process is performed on the time domain information of the denoised audio frame data.

根据本公开的一个或多个实施例，【示例五】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 5] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述对音频帧数据进行分子带处理，包括：In some optional implementation manners of the embodiments of the present disclosure, the performing molecular band processing on the audio frame data includes:

将音频帧数据转化为频域信息后，对所述音频帧数据的频域信息进行分子带处理；After converting the audio frame data into frequency domain information, molecular band processing is performed on the frequency domain information of the audio frame data;

相应的，所述根据其他子带的增益分别对其他子带进行降噪处理，包括：根据其他子带的增益分别对其他子带的频域信息进行降噪处理；Correspondingly, performing noise reduction processing on the other subbands according to the gains of the other subbands includes: performing noise reduction processing on the frequency domain information of the other subbands respectively according to the gains of the other subbands;

所述根据所述第一子带降噪后的频域信息，以及对其他子带进行降噪处理的结果，确定降噪后的音频帧数据，包括：According to the frequency domain information after noise reduction of the first subband and the result of performing noise reduction processing on other subbands, the audio frame data after noise reduction is determined, including:

将所述第一子带降噪后的频域信息，以及其他子带降噪后的频域信息进行拼接，得到所述音频帧数据降噪后的频域信息；将所述音频帧数据降噪后的频域信息转化为时域信息，得到降噪后所述音频数据的帧数据的时域信息。Splicing the denoised frequency domain information of the first subband and the denoised frequency domain information of other subbands to obtain the denoised frequency domain information of the audio frame data; reducing the audio frame data to The denoised frequency domain information is converted into time domain information to obtain the time domain information of the frame data of the audio data after noise reduction.

根据本公开的一个或多个实施例，【示例六】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 6] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述对所述音频帧数据的频域信息进行分子带处理，包括：In some optional implementation manners of the embodiments of the present disclosure, the performing molecular band processing on the frequency domain information of the audio frame data includes:

将所述音频帧数据的频域信息按频率区间进行分组，得到各子带的频域信息；Grouping the frequency domain information of the audio frame data by frequency interval to obtain the frequency domain information of each subband;

相应的，所述将所述第一子带降噪后的频域信息，以及其他子带降噪后的频域信息进行拼接，包括：Correspondingly, splicing the denoised frequency domain information of the first subband and the denoised frequency domain information of other subbands includes:

将所述第一子带降噪后的频域信息，以及其他子带降噪后的频域信息按频率区间进行拼接。The frequency domain information after noise reduction of the first subband and the frequency domain information after noise reduction of other subbands are spliced according to frequency intervals.

根据本公开的一个或多个实施例，【示例七】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 7] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述降噪模型的训练方式，包括：In some optional implementation manners of the embodiments of the present disclosure, the training manner of the noise reduction model includes:

获取与所述第一子带的频率区间相同的样本帧数据，以及所述样本帧数据的目标频域信息；Obtain the same sample frame data as the frequency interval of the first subband, and the target frequency domain information of the sample frame data;

利用所述样本帧数据的频域信息，以及所述目标频域信息，对所述降噪模型进行训练，直至所述降噪模型收敛。Using the frequency domain information of the sample frame data and the target frequency domain information, the noise reduction model is trained until the noise reduction model converges.

根据本公开的一个或多个实施例，【示例八】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 8] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述基于所述第一子带的增益，确定除所述第一子带外的其他子带的增益，包括：In some optional implementation manners of the embodiments of the present disclosure, the determining, based on the gain of the first subband, the gain of other subbands except the first subband includes:

综合所述第一子带的频率区间中，预设频点值到所述区间最大值的平均增益，以及所述降噪模型输出的所述第一子带属于预设类别的判决概率，确定除所述第一子带外的其他子带的增益。In the frequency interval of the first subband, the average gain from the preset frequency point value to the maximum value of the interval, and the judgment probability that the first subband outputted by the noise reduction model belongs to the preset category is determined. Gains of subbands other than the first subband.

根据本公开的一个或多个实施例，【示例九】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 9] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述综合所述第一子带的频率区间中，预设频点值到所述区间最大值的平均增益，以及所述降噪模型输出的所述第一子带属于预设类别的判决概率，确定除所述第一子带外的其他子带的增益，包括：In some optional implementation manners of the embodiments of the present disclosure, in the frequency range of the integrated first subband, the average gain from the preset frequency point value to the maximum value of the range, and the output of the noise reduction model The decision probability that the first subband belongs to a preset category, and the gain of other subbands other than the first subband is determined, including:

将所述第一子带的频率区间中，预设频点值到所述区间最大值的平均增益，作为第一增益因子；Taking the average gain from the preset frequency point value to the maximum value of the interval in the frequency interval of the first subband as the first gain factor;

根据所述第一增益因子，以及所述降噪模型输出的所述第一子带属于预设类别的判决概率，确定除所述第一子带外的其他子带属于预设类别的判决概率；According to the first gain factor and the judgment probability that the first subband belongs to a preset category output by the noise reduction model, determine the judgment probability that other subbands except the first subband belong to a preset category ;

根据其他子带属于预设类别的判决概率，确定其他子带的第二增益因子；Determine the second gain factor of the other subbands according to the decision probability that the other subbands belong to the preset category;

综合所述第一增益因子、第二增益因子，以及其他子带属于预设类别的判决概率，确定除所述第一子带外的其他子带的增益。The gains of other subbands except the first subband are determined by combining the first gain factor, the second gain factor, and the decision probabilities that other subbands belong to a preset category.

根据本公开的一个或多个实施例，【示例十】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 10] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，所述各子带的频率区间依次相邻。In some optional implementation manners of the embodiments of the present disclosure, the frequency intervals of the subbands are adjacent in sequence.

根据本公开的一个或多个实施例，【示例十一】提供了一种音频数据的降噪方法，还包括：According to one or more embodiments of the present disclosure, [Example 11] provides a noise reduction method for audio data, further comprising:

在本公开实施例一些可选的实现方式中，音频数据的降噪方法应用于语音数据降噪，相应的，所述区间最大值最小的频率区间，包含人声频率区间。In some optional implementations of the embodiments of the present disclosure, the method for noise reduction of audio data is applied to noise reduction of speech data. Correspondingly, the frequency interval with the smallest maximum value of the interval includes the frequency interval of human voice.

以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned disclosed concept, the technical solutions made of the above-mentioned technical features or Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above-mentioned features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.

此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。Additionally, although operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although the above discussion contains several implementation-specific details, these should not be construed as limitations on the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。Although the subject matter has been described in language specific to structural features and/or logical acts of method, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (14)

1. A method for noise reduction of audio data, comprising:

performing subband processing on audio frame data, wherein frequency intervals of all subbands are different, and a subband corresponding to a frequency interval with the minimum interval maximum value is used as a first subband;

inputting the frequency domain information of the first sub-band into a noise reduction model, so that the noise reduction model outputs the frequency domain information of the first sub-band after noise reduction and the gain of the first sub-band;

determining gains of other sub-bands except the first sub-band based on the gain of the first sub-band, and respectively carrying out noise reduction processing on the other sub-bands according to the gains of the other sub-bands;

and determining the audio frame data after noise reduction according to the frequency domain information after noise reduction of the first sub-band and the result of noise reduction processing on other sub-bands.

2. The method of claim 1, wherein performing the subband processing on the audio frame data comprises:

performing molecular band processing on the audio frame data in a time domain;

correspondingly, the performing noise reduction processing on the other subbands according to the gains of the other subbands includes: respectively carrying out noise reduction processing on the time domain information of other sub-bands according to the gains of the other sub-bands;

determining the audio frame data subjected to noise reduction according to the frequency domain information subjected to noise reduction of the first sub-band and the result of noise reduction processing on other sub-bands, including:

and converting the frequency domain information subjected to noise reduction of the first sub-band into time domain information, and synthesizing the time domain information subjected to noise reduction of the first sub-band and the time domain information subjected to noise reduction of other sub-bands to obtain the time domain information of the audio frame data subjected to noise reduction.

3. The method of claim 2, wherein performing the subband processing on the audio frame data in the time domain comprises:

performing subband processing on audio frame data through an analysis filter bank, wherein the passband of each filter in the analysis filter bank is different;

correspondingly, the synthesizing the time domain information subjected to noise reduction of the first sub-band and the time domain information subjected to noise reduction of the other sub-bands includes: and synthesizing the time domain information subjected to noise reduction of the first sub-band and the time domain information subjected to noise reduction of other sub-bands by using a comprehensive filter bank.

4. The method of claim 3, further comprising:

before the audio frame data is subjected to the molecular band processing through the analysis filter set, performing down-sampling processing on the audio frame data; or after the audio frame data is subjected to the subband processing through the analysis filter group, performing downsampling processing on the time domain information of the first subband;

correspondingly, before the time domain information subjected to noise reduction of the first sub-band and the time domain information subjected to noise reduction of other sub-bands are synthesized through the comprehensive filter bank, the time domain information subjected to noise reduction of the first sub-band and the time domain information subjected to noise reduction of other sub-bands are subjected to up-sampling processing; or after synthesizing the time domain information of the first sub-band after noise reduction and the time domain information of other sub-bands after noise reduction through the comprehensive filter bank, performing upsampling processing on the time domain information of the audio frame data after noise reduction.

5. The method of claim 1, wherein performing the subband processing on the audio frame data comprises:

after converting audio frame data into frequency domain information, performing molecular band processing on the frequency domain information of the audio frame data;

correspondingly, the performing noise reduction processing on the other subbands according to the gains of the other subbands includes: respectively carrying out noise reduction processing on the frequency domain information of other sub-bands according to the gains of the other sub-bands;

determining the audio frame data subjected to noise reduction according to the frequency domain information subjected to noise reduction of the first sub-band and the result of noise reduction processing on other sub-bands, including:

splicing the frequency domain information subjected to noise reduction of the first sub-band and the frequency domain information subjected to noise reduction of other sub-bands to obtain the frequency domain information subjected to noise reduction of the audio frame data; and converting the frequency domain information of the audio frame data after noise reduction into time domain information to obtain the time domain information of the frame data of the audio data after noise reduction.

6. The method of claim 5, wherein the performing the subband processing on the frequency domain information of the audio frame data comprises:

grouping the frequency domain information of the audio frame data according to frequency intervals to obtain frequency domain information of each sub-band;

correspondingly, the splicing the frequency domain information after the noise reduction of the first sub-band and the frequency domain information after the noise reduction of the other sub-bands comprises:

and splicing the frequency domain information subjected to noise reduction of the first sub-band and the frequency domain information subjected to noise reduction of the other sub-bands according to a frequency interval.

7. The method according to claim 1, wherein the training mode of the noise reduction model comprises:

acquiring sample frame data which is the same as the frequency interval of the first sub-band and target frequency domain information of the sample frame data;

and training the noise reduction model by using the frequency domain information of the sample frame data and the target frequency domain information until the noise reduction model is converged.

8. The method of claim 1, wherein determining gains for subbands other than the first subband based on the gain for the first subband comprises:

and in the frequency interval of the first sub-band, the average gain from a preset frequency point value to the maximum value of the interval and the decision probability of the first sub-band output by the noise reduction model belonging to a preset category are integrated to determine the gains of other sub-bands except the first sub-band.

9. The method according to claim 8, wherein the synthesizing of the average gain from a preset frequency point value to the maximum value of the interval in the frequency interval of the first sub-band and the decision probability that the first sub-band output by the noise reduction model belongs to a preset class to determine the gains of the sub-bands except the first sub-band comprises:

presetting the average gain from a frequency point value to the maximum value of the interval in the frequency interval of the first sub-band as a first gain factor;

determining the decision probability that the other sub-bands except the first sub-band belong to the preset category according to the first gain factor and the decision probability that the first sub-band output by the noise reduction model belongs to the preset category;

determining second gain factors of other sub-bands according to the decision probability that the other sub-bands belong to the preset category;

and integrating the first gain factor, the second gain factor and the decision probability of other sub-bands belonging to a preset category to determine the gains of other sub-bands except the first sub-band.

10. The method of claim 1, wherein the frequency bins of each subband are sequentially adjacent.

11. A method according to any one of claims 1 to 10, applied to noise reduction of speech data, wherein the frequency interval in which the interval has the smallest maximum value, accordingly, comprises the human voice frequency interval.

12. An apparatus for noise reduction of audio data, comprising:

the molecular band module is used for carrying out molecular band processing on the audio frame data, wherein the frequency intervals of all sub-bands are different, and the sub-band corresponding to the frequency interval with the minimum interval maximum value is used as a first sub-band;

a first denoising module, configured to input the frequency domain information of the first subband into a denoising model, so that the denoising model outputs the frequency domain information of the first subband after denoising and a gain of the first subband;

the second noise reduction module is used for determining the gains of other sub-bands except the first sub-band based on the gain of the first sub-band and respectively carrying out noise reduction processing on the other sub-bands according to the gains of the other sub-bands;

and the noise reduction data determining module is used for determining the audio frame data after noise reduction according to the frequency domain information after noise reduction of the first sub-band and the result of noise reduction processing on other sub-bands.

13. An electronic device, characterized in that the electronic device comprises:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method of noise reduction of audio data as claimed in any of claims 1-11.

14. A storage medium containing computer executable instructions for performing a method of noise reduction of audio data as claimed in any one of claims 1-11 when executed by a computer processor.

Images