CN115410547A - Audio processing method and device, electronic equipment and storage medium - Google Patents
Audio processing method and device, electronic equipment and storage mediumDownload PDFInfo
- Publication number
- CN115410547A CN115410547ACN202211027821.5ACN202211027821ACN115410547ACN 115410547 ACN115410547 ACN 115410547ACN 202211027821 ACN202211027821 ACN 202211027821ACN 115410547 ACN115410547 ACN 115410547A
- Authority
- CN
- China
- Prior art keywords
- audio
- microphone
- audio signal
- response
- audio data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1785—Methods, e.g. algorithms; Devices
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/08—Mouthpieces; Microphones; Attachments therefor
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本公开涉及音频处理技术领域，具体涉及一种音频处理方法、装置、电子设备及存储介质。The present disclosure relates to the technical field of audio processing, and in particular to an audio processing method, device, electronic equipment, and storage medium.
背景技术Background technique
近年来，智能手机、耳机、智能手环等电子设备的各项技术逐渐提升，用户的使用体验得到了大幅提升。例如无线蓝牙耳机能够通过蓝牙进行数据和指令传输，使用户在避免有线耳机的复杂线路的同时，获得与有线耳机相同甚至优于有线耳机的音效体验和控制便捷度。无线蓝牙耳机具有自适应音效补偿功能，能够使用户以不同的佩戴方式均能够享受到极致的音效体验。但是目前无线蓝牙耳机的自适应音效补偿功能的效果还有待提高。In recent years, various technologies of electronic devices such as smartphones, earphones, and smart bracelets have gradually improved, and user experience has been greatly improved. For example, wireless Bluetooth headsets can transmit data and commands through Bluetooth, enabling users to obtain the same sound experience and control convenience as wired headsets or even better than wired headsets while avoiding the complicated wiring of wired headsets. The wireless Bluetooth headset has an adaptive sound compensation function, which enables users to enjoy the ultimate sound experience in different wearing styles. However, the effect of the adaptive sound effect compensation function of the wireless bluetooth headset still needs to be improved.
发明内容Contents of the invention
为克服相关技术中存在的问题，本公开实施例提供一种音频处理方法、装置、电子设备及存储介质，用以解决相关技术中的缺陷。In order to overcome the problems existing in the related art, the embodiments of the present disclosure provide an audio processing method, device, electronic equipment and storage medium to solve the defects in the related art.
根据本公开实施例的第一方面，提供一种音频处理方法，应用于音频播放设备，所述音频播放设备具有扬声器、用于采集环境音频的第一麦克风和用于采集所述扬声器播放的音频的第二麦克风；所述方法包括：According to the first aspect of an embodiment of the present disclosure, there is provided an audio processing method applied to an audio playback device, the audio playback device has a speaker, a first microphone for collecting ambient audio, and a first microphone for collecting audio played by the speaker a second microphone; the method comprising:
在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，其中，所述第一响应关系用于表征所述环境音频与所述噪声音频之间的映射关系，所述噪声音频由所述环境音频传输至所述第二麦克风处而形成；When the speaker plays the first audio data, determine the noise signal according to the first audio signal collected by the first microphone and the first response relationship, wherein the first response relationship is used to characterize the environmental audio and the A mapping relationship between noise audio, the noise audio is formed by transmitting the ambient audio to the second microphone;
根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，其中，所述第二响应关系用于表征所述第二音频信号与所述第一音频数据之间的映射关系；According to the first audio data, the noise signal, and the second audio signal collected by the second microphone, determine a second response relationship, wherein the second response relationship is used to characterize the relationship between the second audio signal and A mapping relationship between the first audio data;
根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，其中，所述目标响应关系用于表征所述第二麦克风采集的音频信号与所述扬声器播放的音频数据之间的目标映射关系。According to the second response relationship and the target response relationship, perform sound effect compensation processing on the second audio data to be played to obtain target audio data, wherein the target response relationship is used to characterize the audio signal collected by the second microphone The target mapping relationship with the audio data played by the speaker.
在一个实施例中，还包括：In one embodiment, also includes:
在所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the speaker is not playing audio data, the first response relationship is determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone.
在一个实施例中，所述根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系，包括：In one embodiment, the determining the first response relationship according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone includes:
根据所述第三音频信号的共轭结果和所述第四音频信号，确定交叉功率谱函数，其中，所述交叉功率谱函数用于表征所述第三音频信号与所述第四音频信号的相关程度；According to the conjugation result of the third audio signal and the fourth audio signal, determine a cross power spectrum function, wherein the cross power spectrum function is used to characterize the relationship between the third audio signal and the fourth audio signal Relevance;
根据所述第三音频信号的共轭结果和所述第三音频信号，确定自功率谱函数，其中，所述自功率谱函数用于表征所述第三音频信号的稳定程度；According to the conjugate result of the third audio signal and the third audio signal, determine an autopower spectrum function, wherein the autopower spectrum function is used to characterize the degree of stability of the third audio signal;
根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系。The first response relationship is determined according to the cross power spectrum function and the autopower spectrum function.
在一个实施例中，所述根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系，包括：In one embodiment, the determining the first response relationship according to the cross power spectrum function and the self power spectrum function includes:
将所述交叉功率谱函数与所述自功率谱函数的比值，确定为所述第一响应关系。A ratio of the cross power spectrum function to the autopower spectrum function is determined as the first response relationship.
在一个实施例中，所述在所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系，包括：In one embodiment, when the speaker is not playing audio data, the first audio signal is determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone. Response relationships, including:
在所述音频播放设备由未被佩戴状态切换至被佩戴状态，且所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the audio playback device is switched from the unworn state to the worn state and the speaker is not playing audio data, according to the third audio signal collected by the first microphone and the first audio signal collected by the second microphone, four audio signals to determine the first response relationship.
在一个实施例中，所述根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，包括：In one embodiment, performing sound effect compensation processing on the second audio data to be played according to the second response relationship and the target response relationship to obtain the target audio data includes:
根据所述第二响应关系和所述目标响应关系，确定补偿函数，其中，所述补偿函数用于表征频点与补偿值之间的映射关系；Determine a compensation function according to the second response relationship and the target response relationship, where the compensation function is used to represent a mapping relationship between frequency points and compensation values;
根据所述补偿函数，对所述第二音频数据进行音效补偿处理，得到所述目标音频数据。Perform sound effect compensation processing on the second audio data according to the compensation function to obtain the target audio data.
在一个实施例中，所述音频播放设备还具有由至少一个滤波器组成的音效补偿器，用于对待播放的音频数据进行音效补偿处理；In one embodiment, the audio playback device also has a sound effect compensator composed of at least one filter, which is used to perform sound effect compensation processing on the audio data to be played;
所述根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，包括：According to the second response relationship and the target response relationship, performing sound effect compensation processing on the second audio data to be played to obtain target audio data includes:
根据所述第二响应关系和所述目标响应关系，对所述音效补偿器中至少一个滤波器的滤波参数进行调整。Adjust the filtering parameters of at least one filter in the sound effect compensator according to the second response relationship and the target response relationship.
在一个实施例中，还包括：In one embodiment, also includes:
根据标定指令，从预先配置的多个响应关系中确定目标响应关系，其中每个响应关系均用于表征所述第二麦克风对所述扬声器播放的音频的频率响应关系。According to the calibration instruction, a target response relationship is determined from a plurality of pre-configured response relationships, wherein each response relationship is used to characterize the frequency response relationship of the second microphone to the audio played by the speaker.
在一个实施例中，还包括：In one embodiment, also includes:
控制所述扬声器对所述目标音频数据进行播放。controlling the speaker to play the target audio data.
在一个实施例中，所述第一麦克风包括前馈麦克风；所述第二麦克风包括反馈麦克风。In one embodiment, the first microphone includes a feed-forward microphone; the second microphone includes a feedback microphone.
根据本公开实施例的第二方面，提供一种音频处理装置，应用于音频播放设备，所述音频播放设备具有扬声器、用于采集环境音频的第一麦克风和用于采集所述扬声器播放的音频的第二麦克风；所述装置包括：According to the second aspect of the embodiments of the present disclosure, there is provided an audio processing device applied to an audio playback device, the audio playback device has a speaker, a first microphone for collecting ambient audio, and a first microphone for collecting audio played by the speaker a second microphone; said means comprising:
噪声模块，用于在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，其中，所述第一响应关系用于表征所述环境音频与所述噪声音频之间的映射关系，所述噪声音频由所述环境音频传输至所述第二麦克风处而形成；A noise module, configured to determine a noise signal according to a first audio signal collected by a first microphone and a first response relationship when the speaker plays the first audio data, wherein the first response relationship is used to characterize the A mapping relationship between ambient audio and the noise audio, the noise audio is formed by transmitting the ambient audio to the second microphone;
响应模块，用于根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，其中，所述第二响应关系用于表征所述第二音频信号与所述第一音频数据之间的映射关系；a response module, configured to determine a second response relationship according to the first audio data, the noise signal, and the second audio signal collected by the second microphone, wherein the second response relationship is used to characterize the a mapping relationship between the second audio signal and the first audio data;
补偿模块，用于根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，其中，所述目标响应关系用于表征所述第二麦克风采集的音频信号与所述扬声器播放的音频数据之间的目标映射关系。A compensation module, configured to perform sound effect compensation processing on the second audio data to be played according to the second response relationship and the target response relationship to obtain target audio data, wherein the target response relationship is used to characterize the second A target mapping relationship between the audio signal collected by the microphone and the audio data played by the speaker.
在一个实施例中，还包括确定模块，用于：In one embodiment, it also includes a determination module for:
在所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the speaker is not playing audio data, the first response relationship is determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone.
在一个实施例中，所述确定模块具体用于：In one embodiment, the determination module is specifically used for:
根据所述第三音频信号的共轭结果和所述第四音频信号，确定交叉功率谱函数，其中，所述交叉功率谱函数用于表征所述第三音频信号与所述第四音频信号的相关程度；According to the conjugation result of the third audio signal and the fourth audio signal, determine a cross power spectrum function, wherein the cross power spectrum function is used to characterize the relationship between the third audio signal and the fourth audio signal Relevance;
根据所述第三音频信号的共轭结果和所述第三音频信号，确定自功率谱函数，其中，所述自功率谱函数用于表征所述第三音频信号的稳定程度；According to the conjugate result of the third audio signal and the third audio signal, determine an autopower spectrum function, wherein the autopower spectrum function is used to characterize the degree of stability of the third audio signal;
根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系。The first response relationship is determined according to the cross power spectrum function and the autopower spectrum function.
在一个实施例中，所述确定模块用于根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系时，具体用于：In one embodiment, when the determining module is configured to determine the first response relationship according to the cross power spectrum function and the autopower spectrum function, it is specifically configured to:
将所述交叉功率谱函数与所述自功率谱函数的比值，确定为所述第一响应关系。A ratio of the cross power spectrum function to the autopower spectrum function is determined as the first response relationship.
在一个实施例中，所述确定模块具体用于：In one embodiment, the determination module is specifically used for:
在所述音频播放设备由未被佩戴状态切换至被佩戴状态，且所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the audio playback device is switched from the unworn state to the worn state and the speaker is not playing audio data, according to the third audio signal collected by the first microphone and the first audio signal collected by the second microphone, four audio signals to determine the first response relationship.
在一个实施例中，所述补偿模块具体用于：In one embodiment, the compensation module is specifically used for:
根据所述第二响应关系和所述目标响应关系，确定补偿函数，其中，所述补偿函数用于表征频点与补偿值之间的映射关系；Determine a compensation function according to the second response relationship and the target response relationship, where the compensation function is used to represent a mapping relationship between frequency points and compensation values;
根据所述补偿函数，对所述第二音频数据进行音效补偿处理，得到所述目标音频数据。Perform sound effect compensation processing on the second audio data according to the compensation function to obtain the target audio data.
在一个实施例中，所述音频播放设备还具有由至少一个滤波器组成的音效补偿器，用于对待播放的音频数据进行音效补偿处理；In one embodiment, the audio playback device also has a sound effect compensator composed of at least one filter, which is used to perform sound effect compensation processing on the audio data to be played;
所述补偿模块具体用于：The compensation module is specifically used for:
根据所述第二响应关系和所述目标响应关系，对所述音效补偿器中至少一个滤波器的滤波参数进行调整。Adjust the filtering parameters of at least one filter in the sound effect compensator according to the second response relationship and the target response relationship.
在一个实施例中，还包括标定模块，用于：In one embodiment, a calibration module is also included for:
根据标定指令，从预先配置的多个响应关系中确定目标响应关系，其中每个响应关系均用于表征所述第二麦克风对所述扬声器播放的音频的频率响应关系。According to the calibration instruction, a target response relationship is determined from a plurality of pre-configured response relationships, wherein each response relationship is used to characterize the frequency response relationship of the second microphone to the audio played by the speaker.
在一个实施例中，还包括播放模块，用于：In one embodiment, it also includes a playback module for:
控制所述扬声器对所述目标音频数据进行播放。controlling the speaker to play the target audio data.
在一个实施例中，所述第一麦克风包括前馈麦克风；所述第二麦克风包括反馈麦克风。In one embodiment, the first microphone includes a feed-forward microphone; the second microphone includes a feedback microphone.
根据本公开实施例的第三方面，提供一种电子设备，所述电子设备包括存储器、处理器，所述存储器用于存储可在处理器上运行的计算机指令，所述处理器用于在执行所述计算机指令时实现第一方面所述的音频处理方法。According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, the electronic device includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to execute the The audio processing method described in the first aspect is implemented when the computer instructions are used.
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现第一方面所述的方法。According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.
本公开的实施例提供的技术方案可以包括以下有益效果：The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:
本公开实施例所提供的音频处理方法，首先在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，然后根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，最后根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据。由于确定用于表征所述第二音频信号与第一音频数据之间的映射关系的第二响应关系时，在利用第一音频数据和第二麦克风采集的第二音频信号的基础上，还结合了环境音频传输至第二麦克风处所形成的噪声音频，从而能够避免环境音频所形成的噪声音频对第二响应关系的干扰，提高了第二响应关系的准确性，以及目标音频数据的音效补偿效果。In the audio processing method provided by the embodiments of the present disclosure, firstly, when the speaker plays the first audio data, the noise signal is determined according to the first audio signal collected by the first microphone and the first response relationship, and then the noise signal is determined according to the first The audio data, the noise signal, and the second audio signal collected by the second microphone are used to determine a second response relationship, and finally, according to the second response relationship and the target response relationship, the second audio data to be played is performed Sound effect compensation processing to obtain target audio data. When determining the second response relationship used to characterize the mapping relationship between the second audio signal and the first audio data, on the basis of using the first audio data and the second audio signal collected by the second microphone, combined with The noise audio formed by the transmission of the ambient audio to the second microphone can avoid the interference of the noise audio formed by the ambient audio on the second response relationship, improve the accuracy of the second response relationship, and the sound compensation effect of the target audio data .
附图说明Description of drawings
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明的实施例，并与说明书一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.
图1是本公开一示例性实施例示出的音频处理方法的流程图；FIG. 1 is a flowchart of an audio processing method shown in an exemplary embodiment of the present disclosure;
图2是本公开一示例性实施例示出的音频播放设备的结构示意图；Fig. 2 is a schematic structural diagram of an audio playback device shown in an exemplary embodiment of the present disclosure;
图3是本公开一示例性实施例示出的音频处理装置的结构示意图；Fig. 3 is a schematic structural diagram of an audio processing device shown in an exemplary embodiment of the present disclosure;
图4是本公开一示例性实施例示出的电子设备的结构框图。Fig. 4 is a structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
具体实施方式Detailed ways
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in the present disclosure is for the purpose of describing particular embodiments only, and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present disclosure, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."
近年来，智能手机、耳机、智能手环等电子设备的各项技术逐渐提升，用户的使用体验得到了大幅提升。例如无线蓝牙耳机能够通过蓝牙进行数据和指令传输，使用户在避免有线耳机的复杂线路的同时，获得与有线耳机相同甚至优于有线耳机的音效体验和控制便捷度。无线蓝牙耳机具有自适应音效补偿功能，能够使用户以不同的佩戴方式均能够享受到极致的音效体验。但是目前无线蓝牙耳机的自适应音效补偿功能的效果还有待提高。In recent years, various technologies of electronic devices such as smartphones, earphones, and smart bracelets have gradually improved, and user experience has been greatly improved. For example, wireless Bluetooth headsets can transmit data and commands through Bluetooth, enabling users to obtain the same sound experience and control convenience as wired headsets or even better than wired headsets while avoiding the complicated wiring of wired headsets. The wireless Bluetooth headset has an adaptive sound compensation function, which enables users to enjoy the ultimate sound experience in different wearing styles. However, the effect of the adaptive sound effect compensation function of the wireless bluetooth headset still needs to be improved.
基于此，第一方面，本公开至少一个实施例提供了一种音频处理方法，请参照附图1，其示出了该方法的流程，包括步骤S101至步骤S104。Based on this, in a first aspect, at least one embodiment of the present disclosure provides an audio processing method. Please refer to FIG. 1 , which shows a flow of the method, including steps S101 to S104.
其中，该方法可以应用于音频播放设备。音频播放设备可以为有线耳机、无线蓝牙耳机等耳机，音频播放设备的佩戴形式可以为入耳式、半入耳式、头戴式等。Wherein, the method can be applied to an audio playback device. The audio playback device can be earphones such as wired earphones and wireless bluetooth earphones, and the wearing form of the audio playback device can be in-ear, semi-in-ear, head-mounted, etc.
请参照附图2，其以无线蓝牙耳机为例示出了音频播放设备的结构。从附图2中可以看出，音频播放设备包括外壳、和设于外壳内的扬声器、用于采集环境音频的第一麦克风、用于采集所述扬声器播放的音频的第二麦克风、用于对待播放的音频数据进行音效补偿处理的音效补偿器、以及分别与上述扬声器、第一麦克风、第二麦克风、音效补偿器电连接的芯片。Please refer to accompanying drawing 2, which shows the structure of the audio playback device by taking the wireless bluetooth earphone as an example. As can be seen from Figure 2, the audio playback device includes a shell, and a speaker located in the shell, a first microphone for collecting ambient audio, a second microphone for collecting the audio played by the speaker, and a second microphone for treating A sound effect compensator for performing sound effect compensation processing on the played audio data, and a chip electrically connected to the speaker, the first microphone, the second microphone, and the sound effect compensator respectively.
其中，用户佩戴音频播放设备时，外壳与耳朵内壁贴合从而形成耦合腔，外壳在朝向耦合腔内部(即用户耳道的内部)的位置设有通孔，扬声器朝向耦合腔内部，其播放的音频能够通过上述通孔传输至耦合腔内。Wherein, when the user wears the audio playback device, the shell is attached to the inner wall of the ear to form a coupling cavity. The shell is provided with a through hole facing the inside of the coupling cavity (that is, the inside of the user's ear canal), and the speaker faces the inside of the coupling cavity. Audio can be transmitted into the coupling cavity through the above-mentioned through hole.
其中，第一麦克风可以为前馈麦克风，也就是在用户佩戴音频播放设备时，该麦克风朝向外侧，处于外部环境中，能够采集环境音频；第二麦克风可以为反馈麦克风，也就是用户佩戴音频播放设备时，该麦克风位于扬声器与上述通孔之间，能够采集扬声器播放的音频。Wherein, the first microphone can be a feed-forward microphone, that is, when the user wears an audio playback device, the microphone faces outside, is in an external environment, and can collect ambient audio; the second microphone can be a feedback microphone, that is, the user wears an audio playback device. When the device is installed, the microphone is located between the speaker and the above-mentioned through hole, and can collect the audio played by the speaker.
其中，音效补偿器可以为至少一个滤波器组成的滤波器组。Wherein, the sound effect compensator may be a filter bank composed of at least one filter.
示例性的，该方法可以应用于上述芯片。Exemplarily, the method can be applied to the above chip.
示例性的，该方法可以应用于用户佩戴音频播放设备的场景下。Exemplarily, this method can be applied to a scenario where a user wears an audio playback device.
在步骤S101中，在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，其中，所述第一响应关系用于表征所述环境音频与所述噪声音频之间的映射关系，所述噪声音频由所述环境音频传输至所述第二麦克风处而形成。In step S101, when the speaker plays the first audio data, a noise signal is determined according to the first audio signal collected by the first microphone and the first response relationship, wherein the first response relationship is used to characterize the A mapping relationship between ambient audio and the noise audio, where the noise audio is formed by transmitting the ambient audio to the second microphone.
其中，第一音频数据可以为音频播放设备当前播放的音频数据流中的部分数据，例如按照预设周期(例如以时间计量或者以帧数计量的周期)从音频数据流中截取的数据。该音频数据流可以为音频播放设备所连接的终端设备实时发送至音频播放设备的芯片，再由芯片实时控制音效补偿器对数据进行补偿，并实时控制扬声器进行播放。Wherein, the first audio data may be part of data in the audio data stream currently played by the audio playback device, for example, data intercepted from the audio data stream according to a preset period (for example, a period measured by time or frame number). The audio data stream can be sent to the chip of the audio playback device in real time by the terminal device connected to the audio playback device, and then the chip controls the sound effect compensator to compensate the data in real time, and controls the speaker to play in real time.
可以理解的是，本步骤可以在音频播放设备播放该音频数据流的过程中实时运行，也就是该音频数据流中每个预设周期的数据均作为第一音频数据；或者在音频播放设备播放该音频数据流的过程中按照一定频率运行，也就是该音频数据流中多个预设周期的数据中只有部分作为第一音频数据。It can be understood that this step can be executed in real time during the audio playback device playing the audio data stream, that is, the data of each preset period in the audio data stream is used as the first audio data; or played by the audio playback device The process of the audio data stream runs at a certain frequency, that is, only part of the data of multiple preset periods in the audio data stream is used as the first audio data.
用户佩戴音频播放设备的情况下，外壳会将环境音频隔绝于耦合腔之外；但是上述隔绝作用不会达到百分之百，因此部分环境音频还是会从外壳与耳朵内壁间的缝隙泄露进耦合腔形成噪音音频，这部分噪音音频会由第二麦克风采集并误认为是扬声器所发出的音频。本步骤所确定的噪声信号就是对上述噪音音频的预测。第一响应关系可以为第一麦克风(前馈麦克风)至第二麦克风(反馈麦克风)的初级路径响应函数，用于评估环境音频泄漏进耦合腔而被第二麦克风采集到的噪音信号，也就是用于表征第二麦克风对环境音频的频率响应关系(例如频响曲线)，即所述环境音频与所述噪声音频之间的映射关系。When the user wears an audio playback device, the shell will isolate the ambient audio from the coupling cavity; however, the above-mentioned isolation effect will not reach 100%, so part of the ambient audio will still leak into the coupling cavity from the gap between the shell and the inner wall of the ear to form noise Audio, this part of the noise audio will be picked up by the second microphone and mistaken for the audio from the speaker. The noise signal determined in this step is the prediction of the above noise audio. The first response relationship may be a primary path response function from the first microphone (feedforward microphone) to the second microphone (feedback microphone), which is used to evaluate the noise signal that the ambient audio leaks into the coupling cavity and is collected by the second microphone, that is It is used to characterize the frequency response relationship (for example, frequency response curve) of the second microphone to the environmental audio, that is, the mapping relationship between the environmental audio and the noise audio.
示例性的，可以按照下述确定噪音信号Noiseest：Exemplarily, the noise signal Noiseest can be determined as follows:
Noiseest＝TFpri×XFFNoiseest = TFpri × XFF
其中，TFpri为第一响应关系的频域表示，其包括每帧数据的每个频点的响应值(即增益值)；xFF为第一音频信号的分帧结果，XFF和XFF[m，ω]表示的xFF经过短时傅里叶变换(Short-Time Fourier Transform，STFT)后的频域信号，ω为窗函数。Among them, TFpri is the frequency domain representation of the first response relationship, which includes the response value (i.e. gain value) of each frequency point of each frame of data; xFF is the framing result of the first audio signal, XFF and XFF [m, ω] represents the frequency domain signal of xFF after short-time Fourier transform (Short-Time Fourier Transform, STFT), and ω is a window function.
在步骤S102中，根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，其中，所述第二响应关系用于表征所述第二音频信号与所述第一音频数据之间的映射关系。In step S102, a second response relationship is determined according to the first audio data, the noise signal, and the second audio signal collected by the second microphone, wherein the second response relationship is used to characterize the A mapping relationship between the second audio signal and the first audio data.
第二响应关系可以为扬声器至第二麦克风(反馈麦克风)的次级路径响应函数，用于评估耦合腔模型(即用户的耳道模型)，也就是用于表征第二麦克风对扬声器播放的音频的当前频率响应关系(例如频响曲线)，即第二音频信号与所述第一音频数据之间的映射关系。The second response relationship may be a secondary path response function from the speaker to the second microphone (feedback microphone), which is used to evaluate the coupling cavity model (ie, the user's ear canal model), that is, to characterize the audio played by the second microphone to the speaker The current frequency response relationship (for example, frequency response curve) of , that is, the mapping relationship between the second audio signal and the first audio data.
示例性的，可以按照下述公式确定第二响应关系TFsec：Exemplarily, the second response relationship TFsec may be determined according to the following formula:
其中，TFsec为第二响应关系的频域表示，其包括每帧数据的每个频点的响应值(即增益值)；XFB和XFB[m，ω]表示的xFF经过短时傅里叶变换(Short-Time Fourier Transform，STFT)后的频域信号，ω为窗函数，SPK为第一音频数据的频域表示。Among them, TFsec is the frequency domain representation of the second response relationship, which includes the response value (i.e. gain value) of each frequency point of each frame of data; xFF represented by XFB and XFB [m, ω] passes through For a frequency-domain signal after Short-Time Fourier Transform (STFT), ω is a window function, and SPK is a frequency-domain representation of the first audio data.
在步骤S103中，根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，其中，所述目标响应关系用于表征所述第二麦克风采集的音频信号与所述扬声器播放的音频数据之间的目标映射关系。In step S103, according to the second response relationship and the target response relationship, sound effect compensation processing is performed on the second audio data to be played to obtain target audio data, wherein the target response relationship is used to characterize the second A target mapping relationship between the audio signal collected by the microphone and the audio data played by the speaker.
其中，第二音频数据可以为步骤S101中提到的音频数据流中第一音频数据之后相邻的数据，例如是按照预设周期在第一音频数据之后截取的数据。Wherein, the second audio data may be adjacent data after the first audio data in the audio data stream mentioned in step S101, for example, data intercepted after the first audio data according to a preset period.
其中，目标响应关系可以为标准的响应关系，其能够表征较为标准的耦合腔模型(即用户的耳道模型)，也就是用于表征第二麦克风对扬声器播放的音频的目标频率响应关系(例如频响曲线)，即所述第二麦克风采集的音频信号与所述扬声器播放的音频数据之间的目标映射关系。示例性的，目标响应关系可以为哈曼曲线。Wherein, the target response relationship may be a standard response relationship, which can characterize a relatively standard coupling cavity model (that is, the user's ear canal model), that is, it is used to characterize the target frequency response relationship of the audio played by the second microphone to the speaker (such as frequency response curve), that is, the target mapping relationship between the audio signal collected by the second microphone and the audio data played by the speaker. Exemplarily, the target response relationship may be a Harman curve.
在一个可能的实施例中，可以按照下述方式执行本步骤：In a possible embodiment, this step can be performed in the following manner:
首先，根据所述第二响应关系和所述目标响应关系，确定补偿函数，其中，所述补偿函数用于表征频点与补偿值之间的映射关系，即各个频点的补偿值。示例性的，按照下述确定补偿函数EQ：First, a compensation function is determined according to the second response relationship and the target response relationship, where the compensation function is used to characterize the mapping relationship between frequency points and compensation values, that is, the compensation values of each frequency point. Exemplarily, the compensation function EQ is determined as follows:
EQ{n，k}＝TFtarg et{n，k}-λkTFsec{n，k}EQ{n, k} = TFtarg et {n, k}-λk TFsec {n, k}
其中，EQ{n，k}为补偿函数中第n帧数据的第k个频点的补偿值，TFtarg et{n，k}为目标响应关系中第n帧第k个频点的增益值，TFsec{n，k}为第二响应关系中第n帧第k个频点的增益值，λk为预先设置的第k个频点的权重系数(芯片内存储有每个频点的权重系数)。Among them, EQ{n, k} is the compensation value of the kth frequency point of the nth frame of data in the compensation function, TFtarg et {n, k} is the gain value of the kth frequency point of the nth frame in the target response relationship , TFsec {n, k} is the gain value of the kth frequency point in the nth frame in the second response relationship, and λk is the preset weight coefficient of the kth frequency point (the chip stores each frequency point weight factor).
接下来，根据所述补偿函数，对所述第二音频数据进行音效补偿处理，得到所述目标音频数据。Next, perform sound effect compensation processing on the second audio data according to the compensation function to obtain the target audio data.
其中，示例性的，可以根据所述第二响应关系和所述目标响应关系，对所述音效补偿器中至少一个滤波器的滤波参数进行调整，即根据补偿函数中各频点的补偿值对所述音效补偿器中对应的滤波器的滤波参数进行调整。Wherein, for example, the filtering parameters of at least one filter in the sound effect compensator may be adjusted according to the second response relationship and the target response relationship, that is, according to the compensation value of each frequency point in the compensation function The filter parameters of the corresponding filters in the sound effect compensator are adjusted.
可以在确定目标音频数据后，控制所述扬声器对所述目标音频数据进行播放。After the target audio data is determined, the speaker may be controlled to play the target audio data.
可以理解的是，步骤S101至步骤S103可以是循环执行的，即第N次执行步骤S103所得到的目标音频数据，可以作为第N+1次执行步骤S101时的第一音频数据。It can be understood that step S101 to step S103 may be executed cyclically, that is, the target audio data obtained by executing step S103 for the Nth time may be used as the first audio data when step S101 is executed for the N+1th time.
本公开实施例所提供的音频处理方法，首先在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，然后根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，最后根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据。由于确定用于表征所述第二音频信号与第一音频数据之间的映射关系时，在利用第一音频数据和第二麦克风采集的第二音频信号的基础上，还结合了环境音频传输至第二麦克风处所形成的噪声音频，从而能够避免环境音频所形成的噪声音频对第二响应关系的干扰，提高了第二响应关系的准确性，以及目标音频数据的音效补偿效果。解决了相关技术中只应用反馈麦克风的音频信号和扬声器播放的音频数据进行音效补偿的情况下，噪声造成的音效补偿效果交叉的问题。In the audio processing method provided by the embodiments of the present disclosure, firstly, when the speaker plays the first audio data, the noise signal is determined according to the first audio signal collected by the first microphone and the first response relationship, and then the noise signal is determined according to the first The audio data, the noise signal, and the second audio signal collected by the second microphone are used to determine a second response relationship, and finally, according to the second response relationship and the target response relationship, the second audio data to be played is performed Sound effect compensation processing to obtain target audio data. When determining the mapping relationship between the second audio signal and the first audio data, on the basis of the first audio data and the second audio signal collected by the second microphone, the environmental audio is transmitted to the The noise audio formed at the second microphone can prevent the noise audio formed by the ambient audio from interfering with the second response relationship, thereby improving the accuracy of the second response relationship and the sound compensation effect of the target audio data. It solves the problem that the sound effect compensation effect caused by noise intersects when only the audio signal fed back to the microphone and the audio data played by the speaker are used for sound effect compensation in the related art.
本公开的一些实施例中，还可以在所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。In some embodiments of the present disclosure, when the speaker is not playing audio data, it may be determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone. First responder relationship.
示例性的，可以按照下述方式确定第一响应关系：Exemplarily, the first response relationship may be determined in the following manner:
首先，根据所述第三音频信号的共轭结果和所述第四音频信号，确定交叉功率谱函数，其中，所述交叉功率谱函数用于表征所述第三音频信号与所述第四音频信号的相关程度。示例性的，按照下式确定交叉功率谱函数csd：Firstly, according to the conjugation result of the third audio signal and the fourth audio signal, a cross power spectrum function is determined, wherein the cross power spectrum function is used to characterize the third audio signal and the fourth audio signal signal correlation. Exemplarily, the cross power spectrum function csd is determined according to the following formula:
csd＝X′FF×XFBcsd=X′FF ×XFB
其中，X′FF为第三音频信号XFF的共轭结果，XFB为第四音频信号。Wherein, X'FF is the conjugation result of the third audio signal XFF , and XFB is the fourth audio signal.
接下来，根据所述第三音频信号的共轭结果和所述第三音频信号，确定自功率谱函数，其中，所述自功率谱函数用于表征所述第三音频信号的稳定程度。示例性的，按照下式确定自功率谱函数ssd：Next, according to the conjugation result of the third audio signal and the third audio signal, an autopower spectrum function is determined, where the autopower spectrum function is used to characterize the degree of stability of the third audio signal. Exemplarily, the self-power spectrum function ssd is determined according to the following formula:
ssd＝X′FF×XFFssd=X′FF ×XFF
其中，X′FF为第三音频信号XFF的共轭结果。Wherein, X′FF is the conjugation result of the third audio signal XFF .
最后，根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系。示例性的，将所述交叉功率谱函数与所述自功率谱函数的比值，确定为所述第一响应关系，即按照下式确定第一响应关系TFpri：Finally, the first response relationship is determined according to the cross power spectrum function and the self power spectrum function. Exemplarily, the ratio of the cross power spectrum function to the self power spectrum function is determined as the first response relationship, that is, the first response relationship TFpri is determined according to the following formula:
本实施例中，通过在扬声器未播放音频数据的情况下，利用第一麦克风和第二麦克风采集的音频信号对第一响应关系进行标定，从而使第一响应关系能够更加准确的表征环境音频对第二麦克风的频响关系，也就是使第二响应关系能够更加贴合用户对音频播放设备的佩戴方式(即音频播放设备与用户耳朵内壁的贴合程度)，从而使据此预测的噪声信号能够更加准确。In this embodiment, by using the audio signals collected by the first microphone and the second microphone to calibrate the first response relationship when the speaker is not playing audio data, the first response relationship can more accurately represent the environmental audio pair The frequency response relationship of the second microphone, that is, to make the second response relationship more suitable for the way the user wears the audio playback device (that is, the degree of fit between the audio playback device and the inner wall of the user's ear), so that the predicted noise signal can be more accurate.
可以理解的是，为了使用户每次佩戴音频播放设备时均能够得到较为准确的第一响应关系，可以在所述音频播放设备由未被佩戴状态切换至被佩戴状态，且所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。也就是说，在用户每次佩戴音频播放设备后均进行第一响应关系的标定，从而使得第一响应关系能够贴合用户当前对音频播放设备的佩戴方式(即音频播放设备与用户耳朵内壁的贴合程度)，从而使据此预测的噪声信号能够更加准确，使得用户在使用不同方式佩戴音频播放设备时，均能够得到相同的极致的音效体验。It can be understood that, in order to enable the user to obtain a more accurate first response relationship each time the audio playback device is worn, the audio playback device can be switched from the unworn state to the worn state, and the speaker is not playing In the case of audio data, the first response relationship is determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone. That is to say, after the user wears the audio playback device every time, the first response relationship is calibrated, so that the first response relationship can fit the user's current wearing style of the audio playback device (that is, the distance between the audio playback device and the inner wall of the user's ear). degree of fit), so that the predicted noise signal can be more accurate, so that the user can get the same ultimate sound experience when wearing the audio playback device in different ways.
本公开的一些实施例中，所述音频播放设备(的芯片)内可以预先存储多个响应关系，每个响应关系均用于表征所述第二麦克风对所述扬声器播放的音频的频率响应关系，不同的响应关系作为目标响应关系时，能够使用户获得不同的音效体验。In some embodiments of the present disclosure, multiple response relationships may be pre-stored in (the chip of) the audio playback device, and each response relationship is used to characterize the frequency response relationship of the second microphone to the audio played by the speaker , when different response relationships are used as the target response relationship, the user can obtain different sound experience.
基于此，还可以根据标定指令，从预先配置的多个响应关系中确定目标响应关系，其中每个响应关系均用于表征所述第二麦克风对所述扬声器播放的音频的频率响应关系。Based on this, the target response relationship may also be determined from a plurality of pre-configured response relationships according to the calibration instruction, where each response relationship is used to characterize the frequency response relationship of the second microphone to the audio played by the speaker.
其中，标定指令可以根据用户输入的操作而生成，也就是用户输入针对某个响应关系的选定操作，则生成对应的标定指令，进而确定该响应关系为目标响应关系。Wherein, the calibration instruction can be generated according to the operation input by the user, that is, the user inputs a selected operation for a certain response relationship, and then generates a corresponding calibration instruction, and then determines that the response relationship is a target response relationship.
示例性的，用户可以选定某个响应关系作为目标响应关系后试听音频，音频播放设备会按照附图1所示的方法，提供该目标响应关系下的音效体验，然后用户确定是否选择该响应关系作为目标响应关系。Exemplarily, the user can select a certain response relationship as the target response relationship and listen to the audio, the audio playback device will provide the sound effect experience under the target response relationship according to the method shown in Figure 1, and then the user determines whether to select the response relationship as a target-response relationship.
本实施例中，通过用户的选择操作，可以将用户选定的响应关系作为目标响应关系，从而能够在音频播放设备按照附图1所示的方法播放音频时，提供给用户符合其要求和喜好的音效体验，更加有针对性的提高用户的使用体验。In this embodiment, through the user's selection operation, the response relationship selected by the user can be used as the target response relationship, so that when the audio playback device plays audio according to the method shown in Figure 1, it can provide the user with a response that meets its requirements and preferences. The sound effect experience is more targeted to improve the user experience.
结合上述各个实施例可以看出，本公开提供的音频处理方法，实现了不同噪声环境下的稳定频响计算，从而克服了外界噪声干扰带来的音效补偿不一致问题，强化了算法的稳定性与抗干扰性，让用户在不同的噪声环境下都能享受到一致且优异的音效体验；同时，通过内置多组响应关系(例如频响曲线)，为用户提供了个性化的音效选择，让用户在使用时能够选择自己最喜欢的音效，从而实现个性化的音效补偿，使得具有不同佩戴喜好的用户均能够获得较好的音效体验。Combining the above-mentioned embodiments, it can be seen that the audio processing method provided by the present disclosure realizes stable frequency response calculation under different noise environments, thereby overcoming the problem of inconsistency in sound effect compensation caused by external noise interference, and strengthening the stability and consistency of the algorithm. Anti-interference, so that users can enjoy a consistent and excellent sound experience in different noise environments; at the same time, through built-in multiple sets of response relationships (such as frequency response curves), it provides users with personalized sound options, allowing users When using it, you can choose your favorite sound effect, so as to realize personalized sound effect compensation, so that users with different wearing preferences can obtain a better sound effect experience.
本公开所提供的技术方案中考虑了不同佩戴者之间的耳道模型参数、佩戴方式以及音效喜好的差异性，通过实时计算、自适应调整与适配，实现了不同佩戴者之间均能获得一致的音效体验。且本公开所提供的技术方案具有计算简单、占用资源小、抗干扰性强以及实时自适应适配等优点。In the technical solution provided by the present disclosure, the differences in ear canal model parameters, wearing styles, and sound effect preferences among different wearers are considered, and through real-time calculation, self-adaptive adjustment and adaptation, different wearers can achieve Get a consistent audio experience. Moreover, the technical solution provided by the present disclosure has the advantages of simple calculation, small resource occupation, strong anti-interference, real-time self-adaptation, and the like.
根据本公开实施例的第二方面，提供一种音频处理装置，应用于音频播放设备，所述音频播放设备具有扬声器、用于采集环境音频的第一麦克风和用于采集所述扬声器播放的音频的第二麦克风；请参照附图3，所述装置包括：According to the second aspect of the embodiments of the present disclosure, there is provided an audio processing device applied to an audio playback device, the audio playback device has a speaker, a first microphone for collecting ambient audio, and a first microphone for collecting audio played by the speaker The second microphone; Please refer to accompanying drawing 3, described device comprises:
噪声模块301，用于在所述扬声器播放第一音频数据的情况下，根据第一麦克风采集的第一音频信号和第一响应关系确定噪声信号，其中，所述第一响应关系用于表征所述环境音频与所述噪声音频之间的映射关系，所述噪声音频由所述环境音频传输至所述第二麦克风处而形成；The
响应模块302，用于根据所述第一音频数据、所述噪声信号、以及所述第二麦克风采集的第二音频信号，确定第二响应关系，其中，所述第二响应关系用于表征所述第二音频信号与所述第一音频数据之间的映射；A
补偿模块303，用于根据所述第二响应关系和所述目标响应关系，对待播放的第二音频数据进行音效补偿处理，得到目标音频数据，其中，所述目标响应关系用于表征所述第二麦克风采集的音频信号与所述扬声器播放的音频数据之间的目标映射关系。The
在本公开的一些实施例中，还包括确定模块，用于：In some embodiments of the present disclosure, a determination module is also included for:
在所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the speaker is not playing audio data, the first response relationship is determined according to the third audio signal collected by the first microphone and the fourth audio signal collected by the second microphone.
在本公开的一些实施例中，所述确定模块具体用于：In some embodiments of the present disclosure, the determination module is specifically used to:
根据所述第三音频信号的共轭结果和所述第四音频信号，确定交叉功率谱函数，其中，所述交叉功率谱函数用于表征所述第三音频信号与所述第四音频信号的相关程度；According to the conjugation result of the third audio signal and the fourth audio signal, determine a cross power spectrum function, wherein the cross power spectrum function is used to characterize the relationship between the third audio signal and the fourth audio signal Relevance;
根据所述第三音频信号的共轭结果和所述第三音频信号，确定自功率谱函数，其中，所述自功率谱函数用于表征所述第三音频信号的稳定程度；According to the conjugate result of the third audio signal and the third audio signal, determine an autopower spectrum function, wherein the autopower spectrum function is used to characterize the degree of stability of the third audio signal;
根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系。The first response relationship is determined according to the cross power spectrum function and the autopower spectrum function.
在本公开的一些实施例中，所述确定模块用于根据所述交叉功率谱函数和所述自功率谱函数，确定所述第一响应关系时，具体用于：In some embodiments of the present disclosure, when the determining module is configured to determine the first response relationship according to the cross power spectrum function and the self power spectrum function, it is specifically configured to:
将所述交叉功率谱函数与所述自功率谱函数的比值，确定为所述第一响应关系。A ratio of the cross power spectrum function to the autopower spectrum function is determined as the first response relationship.
在本公开的一些实施例中，所述确定模块具体用于：In some embodiments of the present disclosure, the determination module is specifically used to:
在所述音频播放设备由未被佩戴状态切换至被佩戴状态，且所述扬声器未播放音频数据的情况下，根据所述第一麦克风采集的第三音频信号和所述第二麦克风采集的第四音频信号，确定所述第一响应关系。When the audio playback device is switched from the unworn state to the worn state and the speaker is not playing audio data, according to the third audio signal collected by the first microphone and the first audio signal collected by the second microphone, four audio signals to determine the first response relationship.
在本公开的一些实施例中，所述补偿模块具体用于：In some embodiments of the present disclosure, the compensation module is specifically used for:
根据所述第二响应关系和所述目标响应关系，确定补偿函数，其中，所述补偿函数用于表征频点与补偿值之间的映射关系；Determine a compensation function according to the second response relationship and the target response relationship, where the compensation function is used to represent a mapping relationship between frequency points and compensation values;
根据所述补偿函数，对所述第二音频数据进行音效补偿处理，得到所述目标音频数据。Perform sound effect compensation processing on the second audio data according to the compensation function to obtain the target audio data.
在本公开的一些实施例中，所述音频播放设备还具有由至少一个滤波器组成的音效补偿器，用于对待播放的音频数据进行音效补偿处理；In some embodiments of the present disclosure, the audio playback device also has an audio compensator composed of at least one filter, for performing audio compensation processing on the audio data to be played;
所述补偿模块具体用于：The compensation module is specifically used for:
根据所述第二响应关系和所述目标响应关系，对所述音效补偿器中至少一个滤波器的滤波参数进行调整。Adjust the filtering parameters of at least one filter in the sound effect compensator according to the second response relationship and the target response relationship.
在本公开的一些实施例中，还包括标定模块，用于：In some embodiments of the present disclosure, a calibration module is also included for:
根据标定指令，从预先配置的多个响应关系中确定目标响应关系，其中每个响应关系均用于表征所述第二麦克风对所述扬声器播放的音频的频率响应关系。According to the calibration instruction, a target response relationship is determined from a plurality of pre-configured response relationships, wherein each response relationship is used to characterize the frequency response relationship of the second microphone to the audio played by the speaker.
在本公开的一些实施例中，还包括播放模块，用于：In some embodiments of the present disclosure, a playback module is also included for:
控制所述扬声器对所述目标音频数据进行播放。controlling the speaker to play the target audio data.
在本公开的一些实施例中，所述第一麦克风包括前馈麦克风；所述第二麦克风包括反馈麦克风。In some embodiments of the present disclosure, the first microphone includes a feedforward microphone; the second microphone includes a feedback microphone.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在第一方面有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the above embodiments, the specific manner in which each module executes operations has been described in detail in the embodiment of the method in the first aspect, and will not be described in detail here.
根据本公开实施例的第三方面，请参照附图4，其示例性的示出了一种电子设备的框图。例如，装置400可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等。According to a third aspect of the embodiments of the present disclosure, please refer to FIG. 4 , which exemplarily shows a block diagram of an electronic device. For example, the
参照图4，装置400可以包括以下一个或多个组件：处理组件402，存储器404，电源组件406，多媒体组件408，音频组件410，输入/输出(I/O)的接口412，传感器组件414，以及通信组件416。4,
处理组件402通常控制装置400的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理元件402可以包括一个或多个处理器420来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件402可以包括一个或多个模块，便于处理组件402和其他组件之间的交互。例如，处理部件402可以包括多媒体模块，以方便多媒体组件408和处理组件402之间的交互。The
存储器404被配置为存储各种类型的数据以支持在设备400的操作。这些数据的示例包括用于在装置400上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器404可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电力组件406为装置400的各种组件提供电力。电力组件406可以包括电源管理系统，一个或多个电源，及其他与为装置400生成、管理和分配电力相关联的组件。
多媒体组件408包括在所述装置400和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触控面板(TP)。如果屏幕包括触控面板，屏幕可以被实现为触控屏，以接收来自用户的输入信号。触控面板包括一个或多个触控传感器以感测触控、滑动和触控面板上的手势。所述触控传感器可以不仅感测触控或滑动动作的边界，而且还检测与所述触控或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件408包括一个前置摄像头和/或后置摄像头。当装置400处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。The
音频组件410被配置为输出和/或输入音频信号。例如，音频组件410包括一个麦克风(MIC)，当装置400处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器404或经由通信组件416发送。在一些实施例中，音频组件410还包括一个扬声器，用于输出音频信号。The
I/O接口412为处理组件402和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/
传感器组件414包括一个或多个传感器，用于为装置400提供各个方面的状态评估。例如，传感器组件414可以检测到装置400的打开/关闭状态，组件的相对定位，例如所述组件为装置400的显示器和小键盘，传感器组件414还可以检测装置400或装置400一个组件的位置改变，用户与装置400接触的存在或不存在，装置400方位或加速/减速和装置400的温度变化。传感器组件414还可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件414还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件414还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件416被配置为便于装置400和其他设备之间有线或无线方式的通信。装置400可以接入基于通信标准的无线网络，如WiFi，2G或3G，4G或5G或它们的组合。在一个示例性实施例中，通信部件416经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信部件416还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The
在示例性实施例中，装置400可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述电子设备的音频处理方法。In an exemplary embodiment,
第四方面，本公开在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器404，上述指令可由装置400的处理器420执行以完成上述电子设备的音频处理方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In a fourth aspect, in an exemplary embodiment, the present disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a
本领域技术人员在考虑说明书及实践这里公开的公开后，将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (22)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211027821.5ACN115410547B (en) | 2022-08-25 | 2022-08-25 | Audio processing method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211027821.5ACN115410547B (en) | 2022-08-25 | 2022-08-25 | Audio processing method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115410547Atrue CN115410547A (en) | 2022-11-29 |
| CN115410547B CN115410547B (en) | 2024-10-25 |
Family
ID=84162403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211027821.5AActiveCN115410547B (en) | 2022-08-25 | 2022-08-25 | Audio processing method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115410547B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116456237A (en)* | 2023-03-13 | 2023-07-18 | 小米科技(武汉)有限公司 | Method, device, equipment and storage medium for detecting sound leakage of earphone |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100054381A (en)* | 2008-11-14 | 2010-05-25 | 한양대학교 산학협력단 | Analysis system of transfer path for vibration/noise and analysis method of transfer path for vibration/noise |
| US20140341388A1 (en)* | 2013-05-16 | 2014-11-20 | Apple Inc. | Adaptive audio equalization for personal listening devices |
| US9578432B1 (en)* | 2013-04-24 | 2017-02-21 | Cirrus Logic, Inc. | Metric and tool to evaluate secondary path design in adaptive noise cancellation systems |
| EP3742756A1 (en)* | 2019-05-23 | 2020-11-25 | Beijing Xiaoniao Tingting Technology Co., Ltd | Method and device for detecting wearing state of earphone, earphone, and storage medium |
| WO2021227696A1 (en)* | 2020-05-14 | 2021-11-18 | 华为技术有限公司 | Method and apparatus for active noise reduction |
| US20210409860A1 (en)* | 2020-06-25 | 2021-12-30 | Qualcomm Incorporated | Systems, apparatus, and methods for acoustic transparency |
| WO2022020122A1 (en)* | 2020-07-21 | 2022-01-27 | Starkey Laboratories, Inc. | Ear-wearable device with active noise cancellation system that uses internal and external microphones |
| WO2022048334A1 (en)* | 2020-09-03 | 2022-03-10 | Oppo广东移动通信有限公司 | Testing method and apparatus, earphones, and readable storage medium |
| CN114666702A (en)* | 2022-05-11 | 2022-06-24 | 北京小米移动软件有限公司 | Earphone control method and device, noise reduction earphone and storage medium |
- 2022
- 2022-08-25CNCN202211027821.5Apatent/CN115410547B/enactiveActive
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100054381A (en)* | 2008-11-14 | 2010-05-25 | 한양대학교 산학협력단 | Analysis system of transfer path for vibration/noise and analysis method of transfer path for vibration/noise |
| US9578432B1 (en)* | 2013-04-24 | 2017-02-21 | Cirrus Logic, Inc. | Metric and tool to evaluate secondary path design in adaptive noise cancellation systems |
| US20140341388A1 (en)* | 2013-05-16 | 2014-11-20 | Apple Inc. | Adaptive audio equalization for personal listening devices |
| EP3742756A1 (en)* | 2019-05-23 | 2020-11-25 | Beijing Xiaoniao Tingting Technology Co., Ltd | Method and device for detecting wearing state of earphone, earphone, and storage medium |
| WO2021227696A1 (en)* | 2020-05-14 | 2021-11-18 | 华为技术有限公司 | Method and apparatus for active noise reduction |
| US20210409860A1 (en)* | 2020-06-25 | 2021-12-30 | Qualcomm Incorporated | Systems, apparatus, and methods for acoustic transparency |
| WO2022020122A1 (en)* | 2020-07-21 | 2022-01-27 | Starkey Laboratories, Inc. | Ear-wearable device with active noise cancellation system that uses internal and external microphones |
| WO2022048334A1 (en)* | 2020-09-03 | 2022-03-10 | Oppo广东移动通信有限公司 | Testing method and apparatus, earphones, and readable storage medium |
| CN114666702A (en)* | 2022-05-11 | 2022-06-24 | 北京小米移动软件有限公司 | Earphone control method and device, noise reduction earphone and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116456237A (en)* | 2023-03-13 | 2023-07-18 | 小米科技(武汉)有限公司 | Method, device, equipment and storage medium for detecting sound leakage of earphone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115410547B (en) | 2024-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108430003B (en) | Sound effect compensation method and device, readable storage medium and terminal | |
| JP2018517167A (en) | Method and apparatus for processing audio signals | |
| CN114630239B (en) | Method, device and storage medium for reducing earphone blocking effect | |
| CN114666702B (en) | Earphone control method and device, noise reduction earphone and storage medium | |
| CN113766384A (en) | Method, device and earphone for generating target parameters of energy compensation filter | |
| CN113473304B (en) | Howling suppression method, device, earphone and storage medium | |
| CN107493500A (en) | Multimedia resource player method and device | |
| CN114040285B (en) | Feedforward filter parameter generation method, equipment, headphones and storage medium for headphones | |
| CN115714944A (en) | Audio processing method and device, earphone and storage medium | |
| CN113596662B (en) | Howling sound suppression method, howling sound suppression device, earphones and storage medium | |
| CN115884030A (en) | Noise reduction method, device, earphone device, storage medium and chip | |
| CN114598970A (en) | Audio processing method, device, electronic device and storage medium | |
| CN114786085B (en) | Noise reduction control method and device, noise reduction earphone and storage medium | |
| CN115410547B (en) | Audio processing method, device, electronic equipment and storage medium | |
| CN111698593B (en) | Active noise reduction method and device, and terminal | |
| CN115412632A (en) | Audio data processing method, device, terminal and storage medium | |
| WO2018058331A1 (en) | Method and apparatus for controlling volume | |
| CN114040284B (en) | Noise processing method, noise processing device, terminal and storage medium | |
| CN111736798A (en) | Volume adjustment method, volume adjustment device, and computer-readable storage medium | |
| CN115278441B (en) | Voice detection method, device, earphone and storage medium | |
| EP3941027B1 (en) | Adaptive method and apparatus for intelligent terminal, and terminal | |
| CN115065921A (en) | Method and device for preventing hearing aid from howling | |
| CN114374907A (en) | Sound playback method, device, audio playback device and medium | |
| CN112118502A (en) | Earphone control method, device, equipment and storage medium | |
| CN113495713B (en) | Method and device for adjusting audio parameters of earphones, earphones, and storage media |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |