Summary of the invention
The sound mixing method of software, equipment, system and computer-readable is broadcast live the main purpose of the present invention is to provide a kind ofStorage medium, it is intended to solve traditional sound mixing method and need to carry out the operation such as echo cancellor, cause high pitch to be suppressed, sound quality dropIt is low, the technical issues of audio distortions.
To achieve the above object, the present invention provides a kind of sound mixing methods that software is broadcast live, comprising:
Obtain the audio data sources of the audio accompaniment played；
Stereo process is carried out to the audio data sources and using the collected audio user of microphone.
Optionally, the audio data sources for obtaining the audio accompaniment played, specifically include:
In response to playing the instruction of audio accompaniment, the jump instruction in system audio play function is obtained, wherein the systemSystem audio play function is for executing the function for playing audio accompaniment operation, and the jump instruction is to be previously implanted the audioThe instruction of play function, the jump instruction jump to objective function when the system audio play function is called for making；
According to the jump instruction, the objective function is jumped to from the system audio play function, and described in callingObjective function obtains the audio data sources of the audio accompaniment.
Optionally, described according to the jump instruction, the objective function is jumped to from the system audio play function,And the objective function is called to obtain the audio data sources of the audio accompaniment, it specifically includes:
According to the jump instruction, the objective function is jumped to from the waveout function of windows system；
The objective function is called, audio data sources are obtained by the waveoutwrite function in waveout functionPCM audio data.
Optionally, stereo process is carried out described to the audio data sources and using the collected audio user of microphoneLater, further includes:
The target audio that stereo process obtains is encoded, obtains audio code stream data, and by the audio code fluxionAccording to being pushed to real transport networks.
Optionally, the audio data sources for obtaining the audio accompaniment played, specifically include:
The audio data sources for the audio accompaniment that the accompaniment module and/or accompaniment software for obtaining live streaming software play.
Optionally, stereo process is carried out described to the audio data sources and using the collected audio user of microphoneBefore, further includes:
While playing audio accompaniment using earphone, audio user is acquired by microphone.
To achieve the above object, invention further provides a kind of audio mixing equipment that software is broadcast live, the equipment includesMemory and processor are stored with the computer program that can be run on the processor, the computer on the memoryProgram realizes following method when being executed by the processor:
Obtain the audio data sources of the audio accompaniment played；
Stereo process is carried out to the audio data sources and using the collected audio user of microphone.
Optionally, it is implemented when the computer program is executed by the processor:
In response to playing the instruction of audio accompaniment, the jump instruction in system audio play function is obtained, wherein the systemSystem audio play function is for executing the function for playing audio accompaniment operation, and the jump instruction is to be previously implanted the audioThe instruction of play function, the jump instruction jump to objective function when the system audio play function is called for making；
According to the jump instruction, the objective function is jumped to from the system audio play function, and described in callingObjective function obtains the audio data sources of the audio accompaniment.
Optionally, it is also realized when the computer program is executed by the processor:
The target audio that stereo process obtains is encoded, obtains audio code stream data, and by the audio code fluxionAccording to being pushed to real transport networks.
To achieve the above object, the present invention also provides it is a kind of be broadcast live software mixer system, the system comprises:
Audio data sources acquiring unit: for obtaining the audio data sources of the audio accompaniment played；
Downmixing unit: for being carried out at audio mixing to the audio data sources and using the collected audio user of microphoneReason.
To achieve the above object, described computer-readable to deposit the present invention also provides a kind of computer readable storage mediumComputer program is stored on storage media, the computer program can be executed by one or more processor, as above to realizeA kind of sound mixing method of the live streaming software.
A kind of sound mixing method that software is broadcast live provided by the present invention, the audio accompaniment of broadcasting can be obtained in audio mixingAudio data sources, and stereo process is carried out to the audio data sources and using the collected audio user of microphone.As it can be seen that shouldMethod when carrying out audio mixing, not as traditional sound mixing method acquire loudspeaker output audio and to collected audio intoThe processing such as row echo cancellor, but the audio data sources of the audio accompaniment of broadcasting are directly acquired, therefore, on the one hand avoid acquisitionNoise larger problem when loudspeaker exports audio, the problem of on the other hand avoiding company of collecting wheat audio user, so mixingIt is not necessarily to carry out audio accompaniment the processing such as echo cancellor, ambient noise inhibition during sound, improves the quality of remixed audio.
In addition, the present invention also provides a kind of audio mixing equipment, system and computer readable storage medium that software is broadcast live,Effect corresponds to the above method, and which is not described herein again.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, rightThe present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, notFor limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative workEvery other embodiment obtained is put, shall fall within the protection scope of the present invention.
The description and claims of this application and term " first ", " second ", " third ", " in above-mentioned attached drawingThe (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manageThe data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating hereinOr the sequence other than the content of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended thatCover it is non-exclusive include, for example, containing the process, method, system, product or equipment of a series of steps or units need not limitIn step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, produceThe other step or units of product or equipment inherently.
It should be noted that the description for being related to " first ", " second " etc. in the present invention is used for description purposes only, and cannotIt is interpreted as its relative importance of indication or suggestion or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " theOne ", the feature of " second " can explicitly or implicitly include at least one of the features.In addition, the skill between each embodimentArt scheme can be combined with each other, but must be based on can be realized by those of ordinary skill in the art, when technical solutionWill be understood that the combination of this technical solution is not present in conjunction with there is conflicting or cannot achieve when, also not the present invention claimsProtection scope within.
The present invention provides a kind of sound mixing methods that software is broadcast live.
Referring to Fig. 1, Fig. 1 is a kind of implementation flow chart of sound mixing method embodiment one that software is broadcast live provided by the invention.
As shown in Figure 1, this method specifically includes in embodiment one:
Step S11: the audio data sources of the audio accompaniment of broadcasting are obtained.
The present embodiment, which is mainly used in, connects the scene of wheat in live streaming software, and Lian Mai refers to two or more useFamily in the same place, can not be combined together sound by network the process of singing, chat.In the present embodiment, with twoThe implementation process of the present embodiment is described, for aspect description, the present embodiment divides two users for company's wheat process of userIt is also known as the company wheat user of active user and distal end, the present embodiment is described from the angle of active user.
Specifically, needing to carry out at audio mixing the audio and audio accompaniment of active user in live streaming software running processReason, and then remixed audio is pushed to Lian Maizhe, wherein audio accompaniment may be that the accompaniment software of terminal where active user is broadcastIt puts, it is also possible to what the accompaniment module being broadcast live in software played.In the present embodiment, when software progress audio mixing is broadcast live, pass throughMicrophone acquires the audio of active user, and obtains the audio data sources of the audio accompaniment of broadcasting, it should be noted that companion hereIt plays audio and refers to the audio data that active user one end plays, audio data sources refer to that loudspeaker or earphone play the accompanimentThe source data of audio is not the audio data for record or other operations obtain to the sound of loudspeaker output.In addition,In actual application scenarios, terminal where active user often receives the audio data of real-time network transmission, and according to the soundFrequency data playback Lian Maizhe audio, the audio accompaniment in the present embodiment does not include above-mentioned Lian Maizhe audio.
Step S12: stereo process is carried out to audio data sources and using the collected audio user of microphone.
Audio mixing, which refers to, mixes two-way or reproduction of multiple audio streams, forms the operation of audio stream all the way.It needs to illustrate, in the present embodiment, the collected audio of microphone includes but is not limited to the audio of active user, can also include current useOther audios of environment where family, for example, microphone can when user plays audio accompaniment, Lian Maizhe audio using loudspeakerThe sound, audio accompaniment and Lian Maizhe audio of active user can be collected simultaneously, and when user plays accompaniment tone using earphoneWhen frequency and Lian Maizhe audio, the main collected sound for active user of microphone.As a preferred embodiment, working asPreceding user can play audio accompaniment and Lian Maizhe audio by earphone, in this case, it is believed that microphone is collectedThe only audio of active user, it can thus be avoided carrying out 3A processing to the collected audio of microphone, that is to say, that mixedWhen sound, not only without carrying out 3A processing, audio user can also be handled audio accompaniment to avoid 3A, thus it is guaranteed that accompanimentThe quality of audio and audio user finally realizes the purpose for promoting the quality of remixed audio.
A kind of sound mixing method that software is broadcast live, the audio accompaniment of broadcasting can be obtained in audio mixing provided by the present embodimentAudio data sources, and to audio data sources and the collected audio user of microphone is utilized to carry out stereo process.As it can be seen that the partyMethod the processing such as not acquires the audio of output as traditional sound mixing method and carries out echo cancellor when carrying out audio mixing, butThe audio data sources of the accompaniment of broadcasting are directly found, therefore, noise is larger when on the one hand avoiding acquisition loudspeaker output audioThe problem of, on the other hand also avoid in collection process can together the company of collecting wheat audio user the problem of, so mix processIn without carry out echo cancellor, ambient noise inhibit etc. processing, improve the quality of remixed audio.
Referring to fig. 2, Fig. 2 is a kind of sound mixing method embodiment two that software is broadcast live provided by the invention.
As shown in Fig. 2, in example 2, this method specifically includes:
Step S21: the instruction in response to playing accompaniment obtains the jump instruction in system audio play function.
Wherein, above system audio play function is the function operated in current system for executing broadcasting accompaniment, above-mentionedJump instruction is to be previously implanted the instruction of audio play function, and the jump instruction is for when making system audio play function be calledObjective function is jumped to, is specifically as follows the system plays function by the accompaniment module of live streaming software or accompaniment software transferWhen jump to objective function.
Step S22: according to jump instruction, objective function, and invocation target function are jumped to from system audio play functionObtain the audio data sources of audio accompaniment.
That is, live streaming software obtains the audio data sources of accompaniment using code injection mode.Specifically, live streaming softwareIt can be by calling SetWindowsHookEx function by the accompaniment software of objective function (dll file) injection broadcasting music, moreSpecifically, it is that injection accompaniment software is playing the system audio play function called when accompaniment, makes system audio play functionEntrance be directed toward objective function (goal function is pre-created, and the function of objective function will refer to below).When system audio play function is by accompaniment software transfer, objective function is immediately hopped to, objective function carries out incoming parameterProcessing appropriate, so that it may obtain the PCM data and PCM format of audio data to be played, then pass through process communication for audioData are returned to live streaming software, in order to carry out subsequent stereo process.
Step S23: audio user is acquired by microphone.
Specifically, as shown in figure 3, the present embodiment is suitable for following application scenarios: active user wears earphone, earphone connectionComputer, the audio for enabling active user to hear that the audio accompaniment of computer output and distal end Lian Maizhe are transmitted, therefore export sound notAmbient enviroment is traveled to again, to no longer be collected by microphone.Under this scene, the company wheat user of distal end can be effectively avoidedSound it is again collected after loudspeaker broadcasting, and return to after stereo process the company wheat user of distal end, cause to exist backSound problem.
In practical application scene, even if microphone does not acquire Lian Maizhe audio and audio accompaniment, but it is also possible to collect weekThe noise in collarette border, therefore, it is necessary to carry out ambient noise inhibition, the principle that background noise inhibits to the collected audio of microphoneIt is that the digital signal of real-time sampling is subjected to spectrum analysis, to analyze the intensity and spectrum distribution of ambient noise response, soA filter is designed according to these afterwards, the final frequency spectrum realized according to these background noises and user voice, filter, which allows, works asThe sound spectrum of preceding user passes through, and inhibits to the frequency spectrum of ambient noise, to realize the effect of noise suppression.
Step S24: stereo process is carried out to audio data sources and using the collected audio user of microphone.
It should be noted that the present embodiment do not limit step S24 execute sequence, as long as guaranteeing step S24 at audio mixingReason is completed before being step S25.
Step S25: encoding the target audio that stereo process obtains, and obtains audio code stream data, and by the soundFrequency code flow data is pushed to real transport networks.
The present embodiment specifically can be applied to the computer of windows system, and the interface of Windows system plays music hasWaveout interface, Directsound interface etc., below respectively to upper by taking Hook Waveout interface, Directsound as an exampleStep S22 is stated to be illustrated:
Firstly the need of explanation, Hook be provided in Windows it is a kind of to replace the System Computer interrupted under DOSSystem.After carrying out hook to specific system event, once hook event occurs, the program for carrying out hook to the event willThe notice of system is received, at this moment program can respond the event in first time.
For Hook Waveout interface, the waveOutOpen function of Hook Waveout interface is obtainedWAVEFORMATEX categorical data therefrom obtains the format of audio PCM data, the waveOutWrite of Hook Waveout interfaceFunction obtains LPWAVEHDR categorical data, therefrom obtains audio PCM data and data PCM length.
For Hook Directsound interface, since Directsound component is Com component, Com component Hook modeDifferent, the process with Hook export API are as follows: creation IDirectSound object utilizes IDirectSound Object Creation oneA IDirectSoundBuffer object obtains the empty letter of IDirectSoundBuffer from IDirectSoundBuffer objectNumber table, OnUnlock function address is taken from virtual table, is substituted for new interface function.Hook Directsound interfaceOnUnlock function, LPDIRECTSOUNDBUFFER categorical data can be obtained, the format of audio PCM data is therefrom obtained, takeIt obtains LPVOID data type and obtains audio PCM data.
To sum up, a kind of sound mixing method that software is broadcast live provided by the present embodiment, applied to the company wheat in live streaming softwareScene is all usually in the prior art scheme the sound for acquiring loudspeaker and releasing, and echo cancellor, gain compensation, background are made an uproarAudio mixing is carried out after the processing such as sound inhibition, these audio processings can seriously reduce sound quality, cause user experience poor.And this realityThe audio data sources that example directly finds the audio accompaniment of broadcasting are applied, specifically, the present embodiment is playing accompaniment tone in accompaniment softwareIt is filled with jump instruction in the system audio play function that can be called when frequency, so that accompaniment software plays letter in calling system audioObjective function can be automatic jumped to when number, then objective function can obtain the audio data sources of accompaniment, and audio data sources are passedLive streaming software is given, subsequent stereo process is carried out in order to which software is broadcast live, further, it is also possible to pass through on the basis of above schemeEarphone plays audio accompaniment and Lian Maizhe audio, and microphone is avoided to collect Lian Maizhe audio or companion while acquiring audio userAudio is played, to avoid carrying out 3A processing to the collected audio of microphone, therefore, on the one hand the present embodiment avoids acquisition and raisesNoise is big when sound device exports and there are problems that even wheat audio user, on the other hand avoids to the collected audio of microphoneThe processing such as echo cancellor is carried out, to ensure that the quality of audio accompaniment and audio user, finally improves the product of remixed audioMatter ensure that the company wheat experience of user.
Referring to fig. 4, Fig. 4 is a kind of structural schematic diagram of audio mixing apparatus embodiments that software is broadcast live provided in this embodiment.
As shown in figure 4, the audio mixing equipment 40 of the live streaming software specifically includes: memory 41 and processor 42, the storageThe computer program that can be run on the processor 42 is stored on device 41, the computer program is held by the processor 42Following method is realized when row:
Obtain the audio data sources of the accompaniment played；
Stereo process is carried out to the audio data sources and using the collected audio user of microphone.
In the present embodiment, the audio mixing equipment 40 that software is broadcast live can be PC (Personal Computer, personal electricityBrain), it is also possible to smart phone, tablet computer, palm PC, portable computer, intelligent router, mine machine, network storage are setStandby terminal device.
Wherein, memory 41 include at least a type of readable storage medium storing program for executing, the readable storage medium storing program for executing include flash memory,Hard disk, multimedia card, card-type memory (for example, SD or DX memory etc.), magnetic storage, disk, CD etc..Memory 41It can be the internal storage unit of the audio mixing equipment 40 of live streaming software in some embodiments, such as the audio mixing of the live streaming software is setStandby 40 hard disk.The external storage that memory 41 is also possible to be broadcast live in further embodiments the audio mixing equipment 40 of software is setPlug-in type hard disk that is standby, such as being equipped in the audio mixing equipment 40 of live streaming software, intelligent memory card (Smart Media Card,SMC), secure digital (Secure Digital, SD) blocks, flash card (Flash Card) etc..Further, memory 41 may be used alsoWith the internal storage unit of the audio mixing equipment 40 both including live streaming software or including External memory equipment.Memory 41 not only can be withFor storing the application software and Various types of data that are installed on the audio mixing equipment 40 of live streaming software, such as the code of computer programDeng can be also used for temporarily storing the data that has exported or will export.
Processor 42 can be in some embodiments a central processing unit (Central Processing Unit,CPU), controller, microcontroller, microprocessor or other data processing chips, the program for being stored in run memory 41Code or processing data, such as execute computer program etc..
Bus between memory 41 and processor 42 can be Peripheral Component Interconnect standard (peripheralComponent interconnect, abbreviation PCI) bus or expanding the industrial standard structure (extended industryStandard architecture, abbreviation EISA) bus etc..The bus can be divided into address bus, data/address bus, control alwaysLine etc..Only to be indicated with a thick line in Fig. 4, it is not intended that an only bus or a type of bus convenient for indicating.
As an alternative embodiment, the computer program implements when being executed by the processor 42:
In response to playing the instruction of audio accompaniment, the jump instruction in system audio play function is obtained, wherein the systemSystem audio play function is for executing the function for playing audio accompaniment operation, and the jump instruction is to be previously implanted the audioThe instruction of play function, the jump instruction jump to objective function when the system audio play function is called for making；
According to the jump instruction, the objective function is jumped to from the system audio play function, and described in callingObjective function obtains the audio data sources of the audio accompaniment.
As an alternative embodiment, the computer program is also realized when being executed by the processor 42:
The target audio that stereo process obtains is encoded, obtains audio code stream data, and by the audio code fluxionAccording to being pushed to real transport networks.
Further, the audio mixing equipment 40 that software is broadcast live can also include network interface, and network interface optionally can wrapWireline interface and/or wireless interface (such as WI-FI interface, blue tooth interface) are included, commonly used in setting in the audio mixing of the live streaming softwareIt establishes and communicates to connect between standby 40 and other electronic equipments.
Optionally, the audio mixing equipment 40 of the live streaming software can also include user interface, and user interface may include displayDevice (Display), input unit such as keyboard (Keyboard), optional user interface can also include that the wired of standard connectsMouth, wireless interface.Optionally, in some embodiments, display can be light-emitting diode display, liquid crystal display, touch control type LCDDisplay and OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) touch device etc..Wherein, it showsDevice appropriate can also be known as display screen or display unit, the information handled in the audio mixing equipment 40 for being shown in live streaming softwareAnd for showing visual user interface.
Fig. 4 illustrates only the audio mixing equipment 40 of the live streaming software with memory 41 and processor 42, those skilled in the artMember it is understood that the structure shown in Fig. 4 do not constitute to live streaming software audio mixing equipment 40 restriction, may include than figureShow that less perhaps more components perhaps combine certain components or different component layouts.
Referring to Fig. 5, Fig. 5 is a kind of mixer system embodiment that software is broadcast live provided by the invention.
As shown in figure 5, the system specifically includes:
Audio data sources acquiring unit 51: for obtaining the audio data sources of the audio accompaniment played；
Downmixing unit 52: for being carried out at audio mixing to the audio data sources and using the collected audio user of microphoneReason.
Finally, being deposited on the computer readable storage medium the present invention also provides a kind of computer readable storage mediumComputer program is contained, the computer program can be executed by one or more processor, as described above a kind of to realizeThe sound mixing method of software is broadcast live.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof realIt is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.
The computer program product includes one or more computer instructions.Load and execute on computers the meterWhen calculation machine program instruction, entirely or partly generate according to process or function described in the embodiment of the present invention.The computer canTo be general purpose computer, special purpose computer, computer network or other programmable devices.The computer instruction can be depositedStorage in a computer-readable storage medium, or from a computer readable storage medium to another computer readable storage mediumTransmission, for example, the computer instruction can pass through wired (example from a web-site, computer, server or data centerSuch as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave) mode to another website,Website, computer, server or data center are transmitted.The computer readable storage medium can be computer and can depositAny usable medium of storage either includes that the data storages such as one or more usable mediums integrated server, data center are setIt is standby.The usable medium can be magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or partly leadBody medium (such as solid state hard disk Solid State Disk (SSD)) etc.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed system, device and method can be withIt realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unitIt divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or componentsIt can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown orThe mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unitIt closes or communicates to connect, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unitThe component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multipleIn network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme's.
It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unitIt is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated listMember both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent productWhen, it can store in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantiallyThe all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other wordsIt embodies, which is stored in a storage medium, including some instructions are used so that a computerEquipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the applicationPortion or part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, Random Access Memory), magnetic or disk etc. are various can store journeyThe medium of sequence code.
It should be noted that the serial number of the above embodiments of the invention is only for description, do not represent the advantages or disadvantages of the embodiments.AndThe terms "include", "comprise" herein or any other variant thereof is intended to cover non-exclusive inclusion, so that packetProcess, device, article or the method for including a series of elements not only include those elements, but also including being not explicitly listedOther element, or further include for this process, device, article or the intrinsic element of method.Do not limiting moreIn the case where, the element that is limited by sentence "including a ...", it is not excluded that including process, device, the article of the elementOr there is also other identical elements in method.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hairEquivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skillsArt field, is included within the scope of the present invention.