Specific embodiment
In order to make those skilled in the art better understand the technical solutions in the application, below in conjunction with the application realityThe attached drawing in mode is applied, the technical solution in the application embodiment is clearly and completely described, it is clear that describedEmbodiment is only a part of embodiment of the application, rather than whole embodiments.Based on the embodiment party in the applicationFormula, every other embodiment obtained by those of ordinary skill in the art without making creative efforts, is all answeredWhen the range for belonging to the application protection.
The application provides a kind of voice private chat method of direct broadcasting room, and this method can be applied in system as shown in Figure 1.Referring to Fig. 1, video living transmission system may include voice server, direct broadcast server and client.Wherein, the clientIt can be the terminal device that user uses, in the terminal device, can have net cast software, and the terminal is setThe standby microphone that can above have the voice messaging for including user.In addition, the client can also refer to that the terminal is setThe net cast software of standby middle operation.The net cast software can call the microphone on the terminal device, to includeThe voice messaging of user.The voice server can be used for receiving the voice messaging for the user that each client uploads, and canThese voice messagings according to preset stream media protocol, are converted to voice flow.The direct broadcast server, then can receive masterThe live content that the terminal device broadcast is sent, and the live content can be converted to live streaming audio/video flow.
Referring to Fig. 2, the voice private chat method of direct broadcasting room provided by the present application may comprise steps of.
S1: the private chat request for being directed toward target user is initiated to voice server, so that the voice server is describedPrivate chat channel is established between the initiator and the target user of private chat request.
In the present embodiment, it when user watches live content in direct broadcasting room, can be used with the part in the direct broadcasting roomSame live streaming group is added in family.Different user in same live streaming group can carry out mutual communication by way of voice.ToolBody, user can open the function that voice in group connects wheat.In the case that voice connects the unlatching of wheat function in organizing, the Mike of userWind can acquire the voice messaging of user in real time.The voice messaging of acquisition can be uploaded to voice service by the client of userDevice.In voice server, voice messaging can be converted to the voice flow of user according to preset stream media protocol.This is pre-If stream media protocol for example can be HLS (HTTP Live Streaming, HTTP live stream) agreement.Certainly, this is presetStream media protocol can also be in the light of actual conditions modified.For example, the preset stream media protocol can also be WebRTC(Web Real-Time Communication, page real time communication) agreement.Subsequent, the user that unlatching voice connects wheat function canTo listen to the voice messaging of other users in same live streaming group.At this point, the client of user can initiate number to voice serverAccording to acquisition request.The user identifier of the user can be carried in the data acquisition request.In this way, voice server is receiving thisAfter data acquisition request, user identifier wherein included can be identified.By the user identifier, voice server can be determinedLive streaming group locating for the user identifier, then can by the live streaming group except the user identifier characterization voice flow in addition to otherThe voice flow of user is supplied to the client of the user.On the one hand it can enable the user to hear other in same live streaming groupOn the other hand the Instant audio messages of user also avoid the user from the voice messaging of uppick itself.In this way, user is from voiceThe voice messaging of other users can be used as group chat voice messaging in the same live streaming group obtained at server.The group chat voiceInformation can be obtained by the user in same live streaming group.
In the present embodiment, user carries out group in the live content of viewing direct broadcasting room and with the user of same live streaming groupWhen merely, it can choose with some user in live streaming group or individually linked up with the other users outside live streaming group.At this point, withThe client at family can initiate the private chat request for being directed toward target user to voice server, may include request in private chat requestThe user identifier of initiator and the user identifier of the target user.In this way, voice server is receiving private chat requestAfterwards, the user identifier of both sides can be therefrom extracted, and based on the user identifier extracted, between initiator and target userEstablish private chat channel.The private chat channel can be the channel for being used for transmission private chat voice messaging, the letter transmitted in the private chat channelBreath is independent of each other with the information in group chat channel.
S3: acquiring the private chat voice messaging of the initiator, and the private chat voice messaging of acquisition is uploaded to describedVoice server, so that the voice server provides the private chat voice to the target user by the private chat channelInformation.
In the present embodiment, after establishing private chat channel, in the client of two parties, private chat can occurThe prompt information of state.As shown in figure 3, may include the control of group chat and private chat in the operation interface of user, when private chat frequencyAfter road is established, private chat control can be activated.User can choose the private chat control of the activation in early operation interface.So whenWhen private chat control is selected, the voice messaging of user's typing can be used as private chat voice messaging, rather than group chat voice messaging.Certainly, in practical applications, private chat control can mutually be bound with target user.In this way, as shown in figure 3, when user need with it is moreWhen a user carries out private chat simultaneously, can occur multiple private chat controls, and each private chat control in the interface of the userIn can show the mark (Zhang San, Li Si) of other side, consequently facilitating user distinguishes the object of private chat.Correspondingly, when itIn one or more private chat control it is selected when, the private chat voice messaging of user will be sent to these selected private chat controlsAt the corresponding target user of part.
In the present embodiment, when the initiator of private chat request is under private chat state, the microphone of initiator can be withThe private chat voice messaging of initiator is acquired in the way of in step S1, and the private chat voice messaging of acquisition is uploaded to instituteState voice server.In the private chat voice messaging, the user identifier of the target user can be carried, so as to inform languageWhich user sound server, current private chat voice messaging should be sent to.
In the present embodiment, after voice server receives the private chat voice messaging, the mesh wherein carried can be identifiedThe user identifier of user is marked, so as to the private chat channel for establishing the private chat voice messaging before, is sent to target useFamily.Similarly, target user can reply private chat voice messaging to it after the private chat voice messaging of uppick initiator.ThisSample, voice server can also receive the private chat voice messaging of target user's offer.
In one embodiment, the client of user, can be to private after acquiring the private chat voice messaging of userMerely voice messaging carries out some optimization processings, so that the private chat voice messaging for being uploaded to voice server has higher sound quality.Firstly, client can all remove the sound in private chat voice messaging in addition to voice, so as to reduce environmental noise pairThe influence of voice.Specifically, client can identify the audio frequency characteristics in the private chat voice messaging.The audio frequency characteristics can wrapThe audio frequency characteristics for characterizing voice are included, can also include the audio frequency characteristics for characterizing environmental noise.Typically, voice is pastIt is past to have fixed frequency separation.For example, male sound may be typically located between 64~523Hz, female's sound is usually located at 160~Between 1200Hz.So, this corresponding relationship of voice and fixed frequency separation, can be used as standard voice feature.
In the present embodiment, it when the audio frequency characteristics for including in the private chat voice messaging of identification acquisition, can will be inThe private chat voice messaging of time-domain is converted to frequency domain, and the voice messaging in frequency domain can be and carry out according to frequencyDistribution, and each Frequency point can correspond to certain signal strength.At this point it is possible to be identified from the voice messaging of frequency domainSignal strength reaches the corresponding target frequency of information of specified intensity threshold value.The specified intensity threshold value can be set to human ear canThe intensity of sound obviously heard.In this way, the voice messaging of frequency domain can according to the specified intensity threshold value, be divided into it is multiple fromScattered voice segments, the intensity of voice messaging reaches the specified intensity threshold value in these voice segments.Voice in these voice segmentsInformation can have respective target frequency.These target frequencies can be as the audio for including in the private chat voice messagingFeature.It is then possible to calculate the frequency difference between target frequency frequency corresponding with standard voice feature.Specifically,The center frequency value of the frequency separation of male voice and female voice can be determined respectively.Then, it when calculating frequency difference, can first determineCurrent target frequency and which center frequency value are closer, it is then possible to calculate current target frequency and immediate frequencyFrequency difference between rate central value.The frequency difference can be as between current audio frequency characteristics and standard voice featureDifference value.
In the present embodiment, if the difference value is more than or equal to specified threshold, then it represents that current audio frequency characteristicsDiffer larger with standard voice feature, current audio frequency characteristics are likely to be environmental noise.Therefore, in this case, may be usedTo remove the corresponding information of the audio frequency characteristics from the private chat voice messaging, to filter in the private chat voice messagingComponent environment noise.Wherein, above-mentioned difference value can refer to the absolute value being calculated.The specified threshold can be according to realityBorder situation flexible setting.
In one embodiment, it is contemplated that after handling in a manner mentioned above private chat voice messaging, due toEnvironmental noise is eliminated, then there may be the mute of big section between voice adjacent in private chat voice messaging.FromFor the auditory effect of human ear, that people can be allowed to generate is uncomfortable for big section mute, while people can also be allowed to generate the illusion of communication disruption.MirrorIn this, can big section it is mute in be properly added the lower noise signal of some intensity, to eliminate above-mentioned problem.SpecificallyGround can identify target language segment in the private chat voice messaging, and the intensity value of any information is equal in the target language segmentLower than specified intensity threshold value.Wherein, it is lower than the specified intensity threshold value, shows for the angle of human ear, in the target language segmentVoice messaging can not be gone out by ear recognition, therefore, the target language segment be mute section.At this point it is possible to identify this mute sectionLasting duration show the target voice if the duration of the target language segment is more than or equal to specified duration threshold valueThe duration of Duan Chixu is too long, at this point it is possible to add specified noise signal in the target language segment.The specified noise signalCan be sound of the wind, sound of sea wave etc. will not allow human ear to generate uncomfortable white noise (White Noise).
In one embodiment, private chat voice messaging is carried out to handle it according to above-mentioned removal environmental noise the step ofAfterwards, it is more likely that the part signal in the initial position of normal voice and/or final position can be removed, so as to cause normal languageImperfect or normal voice the starting and/or termination of sound are excessively lofty.In consideration of it, can by the way of signal fitting,It is suitably the starting of voice and final position addition a part fitting information, to solve the problem above-mentioned.It specifically, can be withInitial position and the final position of voice are identified in the private chat voice messaging.Typically, there is language in voice messagingThe place of sound, the waveform that raising and lowering can all occur in the intensity of information can by the identification to information strength in voice messagingTo identify initial position and the final position of voice.At this point it is possible to according to the information waveform for the initial position identified and endThe information waveform that stop bit is set generates corresponding voice fitting information.Voice fitting information and the information of corresponding position splice itAfterwards, continuous waveform can be formed.In this way, adding the voice to match respectively at the initial position and the final positionIt is fitted information, the starting of voice and termination can be enabled more smooth, lofty feeling will not be generated.
In one embodiment, in the private chat voice messaging of the microphone acquisition of user, there may be echo signal,In order to enhance the audio experience of user, the echo signal in the private chat voice messaging can be identified, and from the private chat voiceThe echo signal is removed in information.Specifically, convergence algorithm can be carried out to input signal by sef-adapting filter, madeIt obtains and matches by the shock response that sef-adapting filter obtains with true echo path, so that it is corresponding to obtain echo pathThe estimated value of echo signal.It is then possible to the private chat voice messaging be subtracted to the estimated value of the echo signal, thus from the privateMerely echo signal is removed in voice messaging.
In one embodiment, user might have other people at one's side and speaking in typing private chat voice messaging, fromAnd lead to the sound in the voice messaging of typing there are others.In order to avoid other people sound causes to do to the sound of userIt disturbs, client is after collecting the private chat voice messaging of user, other people language that can will include in the private chat voice messagingMessage breath removal.Specifically, present embodiment can remove other people voice messaging by the method for Application on Voiceprint Recognition.The useIt family can be in advance in the client by a certain number of voice messagings of typing, so that client saves the vocal print of the userFeature.In this way, after client collects the private chat voice messaging of user, can identify the private chat language between net castThe vocal print feature for including in message breath, and the vocal print feature that will identify that is compared with the vocal print feature of the user.If the vocal print feature identified and the vocal print feature of the user are inconsistent, the vocal print feature that can be will identify thatCorresponding information is removed from the private chat voice messaging.Above-mentioned vocal print feature can be and utilize special Application on Voiceprint Recognition groupThe sound wave spectrum that part obtains after analyzing voice messaging.The generation of human language be Body Languages maincenter and vocal organs itBetween a complicated physiology physical process, tongue that people uses in speech, tooth, larynx, lung, nasal cavity is in terms of size and formEveryone is widely different, so the sound wave spectrum of different people is all variant, so that the vocal print feature between different userIt can also be different.Therefore, it is possible to remove the voice messaging of other users by vocal print feature.
S5: the private chat language provided by the target user that the voice server is sent by the private chat channel is receivedMessage breath, and receive the group chat language for the other users that same live streaming group is in the initiator that the voice server is sentMessage breath.
In the present embodiment, voice server again may be by the private chat channel, the private that target user is providedMerely voice messaging is sent to the initiator of private chat request.In addition, voice server can also be by group chat channel, it will be with the hairThe group chat voice messaging for playing other users of the side in same live streaming group is sent to the initiator together.In this way, in initiatorPrivate chat voice messaging and group chat voice messaging can be locally provided simultaneously with.
S7: private chat voice messaging that the target user provides and the group chat voice messaging are synthesized into one voice flowAfterwards, the voice flow of synthesis is played.
In the present embodiment, due to initiator's local reception to two kinds of voice messagings, in order to listen to simultaneously this twoKind of voice messaging needs private chat voice messaging that the target user provides and the group chat voice messaging synthesizing one languageSound stream, and the voice flow after synthesis is played by loudspeaker.In this way, both believing comprising group chat voice in voice flow in post synthesisBreath, and include private chat voice messaging, in addition, in practical applications, the live content of direct broadcasting room can also be added in voice flowVoice messaging so that user when carrying out private chat, will not miss other useful informations in direct broadcasting room.
In one embodiment, due to participating in the voice flow that the user of private chat plays including a large amount of voice messaging,In order to guarantee that user can not hear private chat voice messaging, the client of user can automatically believe the group chat voice in voice flowThe volume of breath is adjusted.Specifically, client can identify the volume of the private chat voice messaging, and according to the institute identifiedVolume is stated, the volume of the group chat voice messaging is adjusted.Wherein, private chat voice messaging and group chat voice messaging initially all may be usedTo be played out according to preset volume, at this point, referring to if the volume of the private chat voice messaging identified is more than or equal toDetermine volume threshold, shows that the user for participating in private chat at this time is illustrating an important content.At this point, in order to not hear the userPrivate chat voice messaging, client can be automatically by the volume adjustment of the group chat voice messaging to lower first volume.SoAfterwards, when the volume of the group chat voice messaging is in first volume, if the sound of the private chat voice messaging identifiedAmount is less than the specified volume threshold, then shows that the user for participating in private chat has completed the elaboration of thing, at this point it is possible to will be describedThe volume adjustment of group chat voice messaging extremely second volume higher than the first above-mentioned volume.For example, second volume can be itVolume when preceding group chat voice messaging normal play.Above-mentioned specified volume threshold, can be the sound than people when normally speakingThe more slightly lower volume value of magnitude.In this way, can suitably turn down group chat voice messaging when thering is user to speak in private chat channelVolume, to guarantee that the private chat voice messaging of user in private chat channel can not heard.In the sound according to private chat voice messagingAmount, after automatically adjusting to the volume of group chat voice messaging, can will the private chat voice messaging and adjust volume afterGroup chat voice messaging merge into a track, and the information after track is merged passes through loudspeaker as the voice flow after synthesisIt plays.
Referring to Fig. 4, the application also provides a kind of client, the client includes:
Private chat request initiating cell, the private chat request for initiating to be directed toward target user to voice server, so that instituteIt states voice server and establishes private chat channel between the initiator and the target user that the private chat is requested；
Private chat voice collecting unit, for acquiring the private chat voice messaging of the initiator, and by the private chat of acquisitionVoice messaging is uploaded to the voice server, so that the voice server is used by the private chat channel to the targetFamily provides the private chat voice messaging；
Voice messaging receiving unit, for receive the voice server by the private chat channel send by the meshThe private chat voice messaging that user provides is marked, and receive that the voice server sends is in same live streaming group with the initiatorOther users group chat voice messaging；
Voice flow synthesis unit, private chat voice messaging and the group chat voice messaging for providing the target userAfter synthesizing one voice flow, the voice flow of synthesis is played.
In one embodiment, the client further include:
Difference value determination unit, for identification audio frequency characteristics in the private chat voice messaging, and the determining audio spyDifference value between sign and standard voice feature；
Voice messaging removal unit, if being more than or equal to specified threshold for the difference value, by the audio frequency characteristicsCorresponding information is removed from the private chat voice messaging.
In one embodiment, the client further include:
Vocal print feature recognition unit, the vocal print feature for including in the private chat voice messaging for identification, and will identify thatThe vocal print feature be compared with the vocal print feature of the initiator；
Voiceprint removal unit, if the vocal print feature for identifying and the vocal print feature of the initiator are differentIt causes, the corresponding information of the vocal print feature that will identify that is removed from the private chat voice messaging.
In one embodiment, the voice flow synthesis unit includes:
Group chat speech volume adjustment module, the volume of the private chat voice messaging for identification, and according to the institute identifiedVolume is stated, the volume of the group chat voice messaging is adjusted；
Track merging module, for merging into the group chat voice messaging after the private chat voice messaging and adjusting volumeOne track, and the information after track is merged is as the voice flow after synthesis.
Referring to Fig. 5, the application also provides a kind of client, the client includes memory and processor, described to depositReservoir when the computer program is executed by the processor, realizes the language of above-mentioned direct broadcasting room for storing computer programSound private chat method.
In the present embodiment, the memory may include the physical unit for storing information, usually by informationIt is stored again with the media using the methods of electricity, magnetic or optics after digitlization.Memory described in present embodiment again may be usedTo include: to store the device of information, such as RAM, ROM in the way of electric energy；The device of information is stored in the way of magnetic energy, it is such as hardDisk, floppy disk, tape, core memory, magnetic bubble memory, USB flash disk；Using the device of optical mode storage information, such as CD or DVD.Certainly, there are also memories of other modes, such as quantum memory, graphene memory etc..
In the present embodiment, the processor can be implemented in any suitable manner.For example, the processor can be withTake such as microprocessor or processor and storage can by (micro-) processor execute computer readable program code (such asSoftware or firmware) computer-readable medium, logic gate, switch, specific integrated circuit (Application SpecificIntegrated Circuit, ASIC), programmable logic controller (PLC) and the form etc. for being embedded in microcontroller.
The concrete function that the device that this specification embodiment provides, memory and processor are realized, can be with this theoryAforementioned embodiments in bright book contrast explanation, and can reach the technical effect of aforementioned embodiments, just no longer superfluous hereIt states.
Therefore technical solution provided by the present application, when user direct broadcasting room watch be broadcast live when, if necessary to and targetUser is individually linked up, then can initiate to request for the private chat of the target user to voice server.Voice serverAfter receiving private chat request, private chat channel can be established between the user and target user.The private chat channel can be used forTransmit the private chat voice messaging between the two users.Meanwhile voice server can also receive each use in same live streaming groupThe group chat voice messaging at family, the group chat voice messaging and private chat voice messaging be received and dispatched by different channels, therefore thatThis will not influence each other, to guarantee the privacy of private chat voice messaging.The initiator of private chat request can pass through local recordSound equipment acquires private chat voice messaging, and uploads the private chat voice messaging to voice server, so that voice server is logicalThe private chat channel established before crossing sends the private chat voice messaging to target user.Similarly, target user can also be to the hairIt plays side and sends private chat voice messaging.Initiator can receive voice after receiving the private chat voice messaging of target user simultaneouslyServer mentions the group chat voice messaging of other users in the same live streaming group sent.The client of initiator can will receivePrivate chat voice messaging and group chat voice messaging are synthesized, to export both voice messagings by the same loudspeaker.ThisSample, user is when carrying out the communication of independent voice with other users, additionally it is possible to hear other voice messagings in direct broadcasting room, not only notWill affect viewing live content, the complexity of user's operation can also be simplified by way of voice-enabled chat, thus improve withThe convenience that other users are linked up.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).SoAnd with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.CauseThis, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable GateArray, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designerVoluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip makerDedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " is patrolledVolume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware DescriptionLanguage)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(RubyHardware Description Language) etc., VHDL (Very-High-Speed is most generally used at presentIntegrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answerThis understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,The hardware circuit for realizing the logical method process can be readily available.
It is also known in the art that other than realizing server in a manner of pure computer readable program code, it is completeEntirely can by by method and step carry out programming in logic come so that server with logic gate, switch, specific integrated circuit, programmableLogic controller realizes identical function with the form for being embedded in microcontroller etc..Therefore this server is considered one kindHardware component, and the structure that the unit for realizing various functions for including in it can also be considered as in hardware component.OrEven, can will be considered as realizing the unit of various functions either the software module of implementation method can be Hardware Subdivision againStructure in part.
As seen through the above description of the embodiments, those skilled in the art can be understood that the application canIt realizes by means of software and necessary general hardware platform.Based on this understanding, the technical solution essence of the applicationOn in other words the part that contributes to existing technology can be embodied in the form of software products, the computer software productIt can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that a computer equipment(can be personal computer, server or the network equipment etc.) executes each embodiment of the application or embodimentMethod described in certain parts.
Each embodiment in this specification is described in a progressive manner, same and similar between each embodimentPart may refer to each other, what each embodiment stressed is the difference with other embodiments.In particular, needleFor the embodiment of client, the introduction control for being referred to the embodiment of preceding method is explained.
The application can describe in the general context of computer-executable instructions executed by a computer, such as programModule.Generally, program module includes routines performing specific tasks or implementing specific abstract data types, programs, objects, groupPart, data structure etc..The application can also be practiced in a distributed computing environment, in these distributed computing environments, byTask is executed by the connected remote processing devices of communication network.In a distributed computing environment, program module can be withIn the local and remote computer storage media including storage equipment.
Although depicting the application by embodiment, it will be appreciated by the skilled addressee that there are many deformations by the applicationWith variation without departing from spirit herein, it is desirable to which the attached claims include these deformations and change without departing from the applicationSpirit.