The content of the invention
The invention provides a kind of server, meeting-place terminal and cloud meeting processing method, it is intended to solves existingThe access capacity that server is depended in technology unduly to server caused by coding and decoding video is low, Consumer's ExperiencePoor the problem of.
In order to solve the above-mentioned technical problem, the invention provides a kind of cloud meeting processing method, including following stepSuddenly：
Receive the local video code stream that each meeting-place terminal is reported；
For each meeting-place terminal, the local video code stream collection issued to it is determined according to default rule respectivelyClose, the local video code stream set at least includes a local video code stream；
The local video code stream that the local video code stream set includes is sent to corresponding described meeting-place wholeEnd.
Optionally, during the local video code stream of each meeting-place terminal of reception, in addition to：
Receive the first audio code stream from each meeting-place terminal；
Each meeting-place terminal will be sent to after the first audio code stream processing of each meeting-place terminal.
Optionally, it is sent to each meeting-place terminal after the first audio code stream processing by each meeting-place terminalIncluding：
First audio code stream is decoded, at least one audio is obtained；
Audio mixing is carried out to the audio；
Audio after the audio mixing is encoded, the second audio code stream is obtained；
Second audio code stream is sent to each meeting-place terminal.
Optionally, the default rule includes：The local video code stream set includes the local of broadcast sourceVideo code flow；
It is described the local video code stream that local video code stream set includes is sent to corresponding meeting-place terminal to include：The local video code stream of the broadcast source is sent to corresponding meeting-place terminal.
Optionally, the default rule also includes：The request of the meeting-place terminal is received, it is described locally to regardFrequency code adfluxion, which is closed, includes the local video code stream of the meeting-place terminal request；
It is described the local video code stream that local video code stream set includes is sent to corresponding meeting-place terminal also to wrapInclude：The local video code stream of the meeting-place terminal request is sent to the meeting-place terminal.
Further, present invention also offers a kind of cloud meeting processing method, comprise the following steps：
The local video code stream of meeting-place terminal is sent to server；
Receive the local video code stream that the local video code stream set that the server issues includes；
The local video code stream is handled, video is obtained.
Optionally, the local video code that the local video code stream set that the reception server is issued includesStream includes：Receive the local video code stream of broadcast source.
Optionally, the local video code that the local video code stream set that the reception server is issued includesStream also includes：Send and ask to the server；Receive the local video code stream of request.
Optionally, it is described the local video code stream of meeting-place terminal is sent to server when, in addition to：WillFirst audio code stream of the meeting-place terminal is sent to the server.
Optionally, first audio code stream of meeting-place terminal is sent to after server described, in addition to：Receive the second audio code stream from server.
Optionally, described that local video code stream is handled, obtaining video includes：To the local videoCode stream is decoded, and obtains at least one sub-video；The sub-video is synthesized, video is obtained.
Further, present invention also offers a kind of server, including：
First receiving module, for receiving the local video code stream that each meeting-place terminal is reported；
Determining module, for for each meeting-place terminal, determining the sheet issued to it according to preset rules respectivelyGround video code flow set, the local video code stream set at least includes a local video code stream；
First sending module, for the local video code stream that the local video code stream set includes to be sent toCorresponding described meeting-place terminal.
Optionally, in addition to the first audio receiving module and audio processing modules；First audio receives mouldBlock is used to receive the first audio code stream from each meeting-place terminal；The audio processing modules are used for will be describedEach meeting-place terminal is sent to after audio code stream processing.
Optionally, the audio processing modules include audio decoder module, mix module, coding module, theOne audio sending module；The audio decoder module is used to be decoded first audio code stream, obtainsAt least one audio；The mix module is used to carry out audio mixing to the audio；The coding module be used for pairAudio after the audio mixing is encoded, and obtains the second audio code stream；The audio sending module is used for instituteState the second audio code stream and be sent to each meeting-place terminal.
Optionally, the default rule includes：The local video code stream set includes the local of broadcast sourceVideo code flow；First sending module includes video sending module, for by the local video of broadcast source codeStream is sent to corresponding meeting-place terminal
Optionally, the default rule also includes：The request of the meeting-place terminal is received, it is described locally to regardFrequency code adfluxion, which is closed, includes the local video code stream of the meeting-place terminal request；The video sending module is additionally operable toThe local video code stream of the meeting-place terminal request is directly forwarded to the meeting-place terminal.
Further, present invention also offers a kind of meeting-place terminal, including：
Second sending module, for the local video code stream of meeting-place terminal to be sent into server；
Second receiving module, for receiving described in the local video code stream set that the server is issued includesLocal video code stream；
Video processing module, for handling the local video code stream, obtains video.
Optionally, second receiving module includes video reception module, is regarded for receiving the local of broadcast sourceFrequency code stream.
Optionally, second receiving module also includes request sending module, for being sent to the serverRequest；The video reception module is additionally operable to receive the local video code stream of request.
Optionally, in addition to the second audio sending module, for by the local video code stream hair of meeting-place terminalWhen giving server, the first audio code stream of the meeting-place terminal is sent to the server.
Optionally, in addition to the second audio receiving module, for receiving the second audio code stream from server.
Optionally, the video processing module includes Video decoding module and synthesis module, the decoder moduleFor being decoded to the local video code stream, at least one sub-video is obtained；The synthesis module is used forThe sub-video is synthesized, video is obtained.
Beneficial effect：
The invention provides a kind of server, meeting-place terminal and cloud meeting processing method, server receives eachThe local video code stream that meeting-place terminal is sent, then for each meeting-place terminal, respectively according to default ruleIt is determined that the local video code stream set issued to it, local video code stream set at least includes a local videoCode stream；The local video code stream that the local video code stream set includes is sent to corresponding described meeting-placeTerminal；The local video code stream that the local video code stream set that meeting-place terminal the reception server is issued includes, andThe local video code stream is handled, video is obtained.Overcome it is of the prior art by server to localThe occupancy of the CPU of server and internal memory is high caused by video code flow is handled, and then causes serverThe problem of access capacity is low, greatly improves the meeting-place terminal access amount of server, and ensure that Consumer's Experience.
Embodiment
Inventive conception is that：By the decoding of the local video code stream completed originally by server, synthesisDeng operation, change by meeting-place terminal to carry out, and can be voluntarily customized according to the demand of different meeting-place terminals,Server eliminates the decoding synthetic operation to local video code stream, so as to save the CPU of server and interiorDeposit resource so that the access capacity of influence server has been changed to the total band of network interface of server by the configuration of serverWidth, so as to greatly improve the access capacity of server.
The embodiment to the present invention is described further below in conjunction with the accompanying drawings.
A kind of cloud meeting processing method is present embodiments provided, Fig. 1 is refer to, including：
S101, receive the local video code stream that each meeting-place terminal is reported；
S102, for each meeting-place terminal, the local video issued to it is determined according to default rule respectivelyCode stream set, local video code stream set at least includes a local video code stream；
S103, the local video code stream that local video code stream set includes is sent to corresponding meeting-place terminal.
When entering to rack meeting, each meeting-place terminal is required to the local video code stream of oneself being sent to serviceDevice.Now, server no longer carries out decoding conjunction to the local video code stream transmitted by each meeting-place terminalInto waiting processing, but for each meeting-place terminal, respectively according to default rule determine to its issue it is localVideo code flow set, and the set of local video code stream at least includes a local video code stream；Then further according toThe local video code stream set issued to meeting-place terminal determined, the sheet that the local video code stream set is includedGround video code flow is sent to each meeting-place terminal, can effectively save the resource of server, allows each meeting-placeThe processing of terminal-pair local video code stream is shared.It is that each meeting-place is whole that server, which sends local video code stream,What end was sent respectively, it there will not be interference between each meeting-place terminal.Due to the meeting-place terminal of participant and more thanOne, therefore, the local video code stream of meeting-place terminal also has a lot；So, the difference in step S103Implication, be exactly that local video code stream needed for each meeting-place terminal is likely to more than one, then just shouldLocal video code stream needed for the terminal of meeting-place is forwarded respectively；Server is by each meeting-place terminal receivedLocal video code stream, untreated direct forwarding is transmitted to the required of each meeting-place terminal and local regardsFrequency code stream is the original local video code flow that each meeting-place terminal is sent to server.Local video code stream set,Refer to that each terminal reports the set of the local video code stream of server, this set is original local regardsThe set of frequency code stream, and included in local video code stream set corresponding to each meeting-place terminal local regardFrequency code stream is probably identical, it is also possible to part identical, or different.
Above-mentioned default rule, can include：By the local video code stream of broadcast source, local video is used asA part in code stream set, the i.e. set of local video code stream include the local video code stream of broadcast source；May be used alsoWith including receiving the request of meeting-place terminal, it will the local video code stream of field terminal request is used as local videoA part in code stream set, the i.e. set of local video code stream include the local video code stream of meeting-place terminal request.
In general, the local video code stream for being sent to some meeting-place terminal should be different from meeting-place endThe local video code stream at end, certainly, the meeting-place terminal local video code stream of oneself are need not to be connect from serverReceive；In cloud meeting, any meeting-place terminal should at least have the picture of a meeting-place terminal, when moreTime can have multiple pictures.Because the meeting-place terminal number of participant is not unique, and each meeting-place terminalIt is not to be required for seeing all meeting-place terminals, some meeting-place terminals, may for the consideration given top priority to what is the most importantThe video of some meeting-place terminal can be only checked, some meeting-place terminals again perhaps to look at that other meeting-place are whole moreThe picture at end and select to check the video of all meeting-place terminals, therefore, server what is desired is that, according to notThe demand of same meeting-place terminal, the local video code stream for those meeting-place terminals that these meeting-place terminals are needed is sent outGive these meeting-place terminals.
In cloud meeting, broadcast source is usually had, broadcast source is also one in the terminal of meeting-place, i.e., whole cloudThe overall situation of meeting is controlled by this meeting-place terminal, therefore, generally requires to regard the local of this meeting-place terminalFrequency code stream is sent to each meeting-place terminal, then, now, the set of local video code stream includes broadcast sourceLocal video code stream, it is necessary to the local video code stream of broadcast source is directly forwarded to other each meeting-place wholeEnd；In addition, broadcast source be also required in a meeting point to see some or some meeting-place terminals, accordingly,Need the local video code stream of these meeting-place terminals being sent to broadcast source；Broadcast source can see other meeting-placeTerminal, can also see broadcast source oneself, and the meeting-place terminal that broadcast source is seen is known as the end that broadcast source is seen；In a cloud meeting, often have the two more special meeting-place terminals, wherein, broadcast source it is localVideo code flow needs to be sent to each meeting-place terminal, and the end that broadcast source is seen needs to be sent to broadcast source.Certainly,The end that broadcast source is seen can change in a cloud meeting, and may be regular change, thisHave no effect on the implementation of this programme.In addition, for various reasons, broadcast source is in itself in once cloud meetingIt is that possible change, also should be by the local video code stream of the broadcast source after change after broadcast source changeIt is sent to each meeting-place terminal.
Each meeting-place terminal in cloud meeting whenever, whether just start, or in a meeting, allIt can send and ask to server, this request at least includes the meeting-place terminal that the meeting-place terminal needs to see,I other words, each meeting-place terminal to server, which is sent, includes the request for the meeting-place terminal that needs are seen；Server is thenAccording to the request, determine that the local video code stream needed for the meeting-place terminal, the i.e. set of local video code stream includeThe local video code stream of meeting-place terminal request, and send corresponding local video code stream to the meeting-place terminal.I other words, each meeting-place terminal should at least receive the local video code stream of broadcast source, further, it is also possible to connectReceive the local video code stream of other meeting-place terminals.
Server accordingly, can also receive each meeting when receiving the local video code stream of each meeting-place terminalFirst audio code stream of field terminal；When receiving the first audio code stream of each meeting-place terminal, really, serviceFirst audio code stream of each meeting-place terminal directly can also be sent to each meeting-place terminal and be handled by device,However, due to audio processing with video processing compared to its occupancy server CPU and internal memory it is relatively low,Therefore it is not necessarily to；In addition, proceed from the reality, due to each with can meeting-place terminal in a meeting and be not required toSee the picture of each meeting-place terminal, but generally require the sound for hearing all meeting-place terminals, soThe meeting that can enable is normally carried out.If meeting-place terminal selectively is received to the first audio code stream stream,The meeting-place terminal can be then caused to miss the speech of other some meeting-place terminals, if the speech missed is especially importantWords, then can directly affect the quality of this meeting.Therefore, transmitted by each meeting-place terminal in the present embodimentThe first audio code stream, server directly handled first audio code stream, is then sent to each meetingField terminal.And the first audio code stream progress processing transmitted by each meeting-place terminal is included：To the first audioCode stream is decoded, and obtains at least one audio；Audio mixing is carried out to the audio；Audio after audio mixing is carried outCoding, obtains the second audio code stream；Second audio code stream is sent to each meeting-place terminal.Wherein, to eachThe first audio decoder and/or the second audio coding that individual meeting-place terminal is sent can include Waveform Codec, parameterAny feasible mode in encoding and decoding and mixed encoding and decoding；Waveform Codec includes PCM (Pulse ModeModulation, pulse code modulation), ADPCM (Adaptive Difference Pulse CodeModulation, adaptive difference pulse code modulation), SB-ADPCM (Subband AdaptiveDifferential Pulse Code Modulation, subband-adaptive difference pulse code) etc. compile solutionCode mode, parametric codec includes LPC (Linear Predictive Coding, linear predictive coding) etc.Code encoding/decoding mode, mixed encoding and decoding include CELPC (Code Excited Linear Predictive Coding,QCELP Qualcomm), VSLPC (Vector Sum Excited Linear Predictive Coding,VSELP), RPE-LTP (Regular Pulse Excited-Long TermPredictive, Regular-Pulse Excitation long-term prediction), LD-CELP (Low Delay-Code Excited LinearPredictive, low delay CELP), MPE (Multi-Pulse Excited, multiple-pulse swashEncourage) etc. code encoding/decoding mode.It is audio to carry out audio mixing (Audio Mixing, be often simply called mix) to audioA step in processing, is the sound a variety of sources, is integrated into a sound.It is each in the present embodimentIndividual audio is from different meeting-place terminals, in mix process, can by the frequency of each audio, dynamic,Tonequality, positioning, reverberation and sound field are individually adjusted, and allow each track to optimize, and are superimposed on again afterwards finalOn finished product.This processing mode, can produce well-bedded audio frequency effect.Audio mixing can be by audio mixing softwareTo be handled.Audio after audio mixing is encoded again, it is possible to be sent to each meeting-place terminal.ServiceDevice can be handled all audios, but in some cases, server can also selectivity to eachThe audio code stream of individual meeting-place terminal is handled, and this is probably that need not be spoken because of the meeting-place terminal, orAllocated by broadcast source, by the way of making a speech in order etc..
A kind of cloud meeting processing method is present embodiments provided, Fig. 2 is refer to, including：
S201, the local video code stream of meeting-place terminal is sent to server；
The local video code stream that the local video code stream set that S202, the reception server are issued includes；
S203, local video code stream is handled, obtain video.
In cloud meeting, each meeting-place terminal for adding cloud meeting should be by respective local video code streamIt is sent to server.Although each Cloud Server is in same cloud meeting, each meeting-place terminal is sentThe process of local video code stream is independent, i.e., each meeting-place terminal is respectively by respective local video code streamServer is sent to, likewise, at least one other meeting-place terminal of each meeting-place terminal the reception serverThe process of local video code stream is also independent, will not be interfered with each other between each meeting-place terminal.
The local video code stream that the local video code stream set that the reception server is issued includes is referred in cloud meetingIn view, server have received the local video code stream that each meeting-place terminal is sent respectively, then by local videoCode stream is sent respectively to each meeting-place terminal；Each meeting-place terminal local video code stream to be received should beThe local video code stream of other meeting-place terminals, and the local video code stream of meeting-place terminal oneself need not be received；ThisThe local video code stream that ground video code flow set includes, it is however generally that, refer in cloud meeting, any meeting-placeTerminal should at least have the picture of a meeting-place terminal, there is multiple pictures when more.Due to the meeting of participantField terminal number is not unique, and each meeting-place terminal is not to be required for seeing all meeting-place terminals,Some meeting-place terminals only may can check the video of some meeting-place terminal, have for the consideration given top priority to what is the most importantMeeting-place terminal again perhaps to looking at the picture of other meeting-place terminals more and selecting to check all meeting-place terminalsVideo, therefore, server what is desired is that, it is according to the demand of different meeting-place terminals, these meeting-place are wholeThe local video code stream of those the meeting-place terminals needed is held to be sent to these meeting-place terminals.Local video code stream collectionClose, refer to that each terminal reports the set of the local video code stream of server, this set is original sheetThe set of ground video code flow, and sheet included in local video code stream set corresponding to each meeting-place terminalGround video code flow is probably identical, it is also possible to part identical, or different.
In cloud meeting, broadcast source is usually had, broadcast source is also one in the terminal of meeting-place, i.e., whole cloudThe overall situation of meeting is controlled by this meeting-place terminal, therefore, generally requires to regard the local of this meeting-place terminalFrequency code stream is sent to each meeting-place terminal, then, now, each meeting-place terminal is accomplished by receiving broadcast sourceLocal video code stream, i.e. local video code stream set includes the local video code stream of broadcast source；Certainly, thisIn receive or as transmitted by server；In addition, broadcast source be also required in a meeting point to see certainIndividual or some meeting-place terminal, accordingly, it is necessary to which the local video code stream of these meeting-place terminals is sent to extensivelyBroadcast source, that is to say, that broadcast source needs to receive the local video code stream at the end that broadcast source is seen；Broadcast source can be seenOther meeting-place terminals, can also see broadcast source oneself, and the meeting-place terminal that broadcast source is seen is known as broadcastThe end that source is seen；In a cloud meeting, the two more special meeting-place terminals are often had, wherein, extensivelyBroadcasting the local video code stream in source needs to be sent to each meeting-place terminal, and the end that broadcast source is seen needs to be sent to extensivelyBroadcast source.Certainly, the end that broadcast source is seen can change in a cloud meeting, and may be regularChange, this has no effect on the implementation of this programme.In addition, for various reasons, broadcast source is in itself oneIt is also to change in secondary cloud meeting, also should be by the broadcast source after change after broadcast source changeLocal video code stream is sent to each meeting-place terminal.
Each meeting-place terminal in cloud meeting whenever, whether just start, or in a meeting, allIt can send and ask to server, this request at least includes the meeting-place terminal that the meeting-place terminal needs to see,I other words, each meeting-place terminal to server, which is sent, includes the request for the meeting-place terminal that needs are seen；Server is thenCorresponding local video code stream is sent to the meeting-place terminal according to the request.The set of local video code stream includes meetingThe local video code stream of field terminal request.In other words, each meeting-place terminal should at least receive broadcast sourceLocal video code stream, further, it is also possible to receive the local video code stream of other meeting-place terminals.
After at least one local video code stream is received, it is possible to which local video code stream is handled, placeThe process of reason includes：Local video code stream is decoded, at least one sub-video is obtained；Then antithetical phrase is regardedFrequency performs synthetic operation, obtains video.The local video code stream that meeting-place terminal is received, including broadcast source,May have multiple, terminal-pair this multiple local video code stream in meeting-place are decoded；The form of the encoding and decoding of videoMainly have following several：H.261, H.263, H.264, using any form.Each is localAfter video code flow decoding, at least one sub-video has been obtained, it is possible to according to the meeting-place terminal wish pair of oneselfEach sub-video is synthesized.The screen layout of video after synthesis can be arbitrary, some conventional conjunctionsScreen layout after can be：First, the sub-video for corresponding to each meeting-place terminal exists according to same sizeIt is uniformly distributed in picture；Second, using one of sub-video as main video, make its picture maximum, it is otherSub-video is as from video, and picture is distributed in around key frame, wherein, main view frequency can be made with broadcast sourceMain view frequency is used as the video of the meeting-place terminal of main presentation in main video, or now cloud meeting；Third,The picture of video after synthesis can be dynamic, and corresponding amplification, shows if whose video pictures who makes a speechMust be than more prominent.It is equal in the present embodiment using any of the above mode or other NM synthesis modesIt is applicable, as long as its user that the video in the meeting-place terminal can be allowed normally to be shown to participant sees.So, the processing procedure of the decoding synthesis of local video code stream is gone to corresponding meeting-place terminal to handle, byBe in the processing of each meeting-place terminal it is independent, not only can be customized according to each meeting-place terminal individual, have veryGood Consumer's Experience, the CPU and internal memory for also reducing server end is used so that server can be accessed moreMany meeting-place terminals, so as to improve the efficiency of meeting.
, can also be by each meeting-place terminal while the local video code stream of meeting-place terminal is sent to serverThe first audio code stream be sent to server.When sending the first audio code stream of each meeting-place terminal, really,Meeting-place terminal can also be handled audio code stream using the processing mode of similar local video code stream, stillAudio code stream has its particularity, and the processing of audio is compared with the processing of video, the service shared by audio frequency processThe CPU and internal memory of device are relatively low, therefore are not necessarily to；In addition, proceeding from the reality, due to each meeting with meetingTerminal in a meeting and need not see the picture of each meeting-place terminal, but generally require and hear all meetingsThe sound of field terminal, could so enable meeting to be normally carried out.Therefore, each meeting-place terminal is by firstAudio code stream is sent to after server, and server is handled it, obtains the second audio code stream, then willSecond audio code stream is sent to each meeting-place terminal.Processing of the server to the first audio code stream is implemented with the aboveIt is consistent in example, repeat no more here.
A kind of cloud meeting processing method is present embodiments provided, Fig. 3 is refer to, including：
Terminal T1-Tn callings in participant meeting-place are added meeting by S301, server；
The promoter of cloud meeting is typically exactly broadcast source, and broadcast source initiates to call by server, sets up cloud meetingView.
S302, each meeting-place terminal T1-Tn add meeting, and by local video code stream, the first audio code streamIssue server；
Each receives the meeting-place terminal of request for conference when adding meeting, just by respective local video code stream,Audio code stream is sent to server.
S303, server determine that Tx meeting-place terminal is conference broadcast source；Ty meeting-place terminal is that broadcast source meeting-place is wholeThe end that end Tx is seen；
Due to each and can meeting-place terminal at least need the picture of broadcast source in a meeting, therefore server is neededIt is broadcast source which, which is determined,；Because broadcast source is the promoter of meeting, this step can also be at the very startCarry out.
The conference broadcast source Tx received local video code stream is handled straight by S304, server without encoding and decodingConnect transparent transmission, be transmitted to the T1-Tn meeting-place terminal of membership；By the local video code stream of the Ty meeting-place terminal receivedIt is transmitted to broadcast source Tx meeting-place terminal.Cloud Server flows into the first audio code that each meeting-place terminal is sent simultaneouslyThe second audio code stream is formed after row decoding, audio mixing re-encoding, then sends back to all meeting-place terminals again；
Server no longer carries out decoding synthesis etc. to the local video code stream transmitted by each meeting-place terminalReason, but the local video of at least one other meeting-place terminal of each meeting-place terminal is directly will differ from respectivelyCode stream is sent to each meeting-place terminal, can effectively save the resource of server, allows each meeting-place terminal-pairThe processing of local video code stream is shared.
The local video code stream of broadcast source should be sent in each other meeting-place terminal, i.e. T1-Tn and removeTx meeting-place terminal；The local video code stream for the end Ty that broadcast source is seen then is sent to broadcast source.
At the same time, server also receives the first audio code stream from each meeting-place terminal, to first soundFrequency code stream carries out decoding process, obtains at least one audio；Audio mixing is carried out to these audios；Then by audio mixingAudio afterwards is encoded, and obtains the second audio code stream, then the second audio code stream is sent into each meeting-place endEnd.
If S305, the request for not receiving meeting-place terminal, continue to keep S304 flow；
The local video code stream of broadcast source has now been sent to each meeting-place terminal, the second sound by serverFrequency code stream has also had transmitted to each meeting-place terminal；If meeting-place terminal does not check the picture of other meeting-place terminalsThe demand in face, then, the step of now server continues with S304 continues cloud meeting.
S306, sentence in the request that meeting-place terminal is sended over, the content of server meeting-place terminal request when receivingThe meeting-place terminal request of breaking which is/the local video code stream of which meeting-place terminal；
When meeting-place terminal wants to check the picture of some meeting-place terminals, meeting-place terminal just sends one to serverRequest, what server then confirmed that the meeting-place terminal wants to see according to the request is which/sheet of which meeting-place terminalGround video code flow.
Corresponding local video code stream is sent to the meeting-place terminal by S307, server.
Server confirms after the local video code stream for the meeting-place terminal that meeting-place terminal needs are seen, it is possible toThe local video code stream is sent to the meeting-place terminal；Afterwards, just by the meeting-place terminal-pair local video codeStream is handled, so as to obtain required video pictures.
A kind of cloud meeting processing method is present embodiments provided, Fig. 4 is refer to, including：
S401, this meeting-place terminal receive membership request；
Now, broadcast source has initiated cloud meeting, and corresponding meeting-place terminal then needs membership.
S402, this meeting-place joining meeting to terminal, and local local video code stream, the first audio code stream are compiledServer is issued after code；
This meeting-place terminal is when adding meeting, just by the local video code stream of this meeting-place terminal, the first audio codeStream is sent to server；Local video code stream and the first audio code stream are all formed by coding, according to eachFrom coded format.
S403, this meeting-place terminal receive the local video code stream and second for the broadcast source that Cloud Server is sended overAudio code stream；The meeting-place terminal list of participant is received simultaneously；
The local video code stream of broadcast source needs each meeting-place terminal to receive, that is to say, that the picture of broadcast sourceEach meeting-place terminal is required for seeing；Server receive the first audio code stream that each meeting-place terminal sends itAfterwards, just the audio code stream is handled, including decoding, audio mixing re-encoding, it is then sent to each meeting-placeTerminal.
This meeting-place terminal except need receive broadcast source local video code stream and the second audio code stream in addition to,The meeting-place terminal situation of participant should be also learnt, so can just decide whether to see other meeting-place terminals.
S404, this meeting-place terminal are handled the local video code stream and the second audio code stream received respectively,Form video and audio；
The local video code stream that meeting-place terminal is now received is the local video code stream of broadcast source；To locally regardingThe processing of frequency code stream includes decoding, then synthesizes；When only one of which local video code stream, without being synthesizedOperation, directly carries out decoding to local video code stream and can be obtained by video；Second audio code stream is eachFirst audio code stream of meeting-place terminal is obtained by server-side processes, and after decoding, this meeting-place terminal can just be obtainedAudio to after each meeting-place terminal synthesis.
If S405, not sending and asking to server, this meeting-place terminal keeps step S403, S404；
If this meeting-place terminal does not have the demand of many pictures, now need only to keep the picture for seeing broadcast source i.e.Can.
S406, this meeting-place terminal to server send request, first determine to need from the meeting-place terminal list of participantSee the picture of those meeting-place terminals；Then request message is issued into server, seen in request comprising needsThe list of meeting-place terminal；
If this meeting-place terminal needs to see the image of other meeting-place terminals, first, it is determined that needing the meeting-place seen wholeWhich end is；Then, the list for embodying these meeting-place terminals is sent to server.
S407, Cloud Server find out corresponding meeting-place from the list of the meeting-place terminal in the request of meeting-place terminalTerminal, and this meeting-place terminal is issued into the local video code stream circulation of these meeting-place terminals；
S408, this meeting-place terminal are decoded respectively after receiving these local video code stream streams, then are synthesized manyPicture, is output on display device.
A kind of server is present embodiments provided, Fig. 5 is refer to, including：
First receiving module 101, for receiving the local video code stream that each meeting-place terminal is reported；
Determining module 105, for for each meeting-place terminal, being determined to issue to it respectively according to default ruleLocal video code stream set, the set of local video code stream at least includes local video code stream；
First sending module 102, for the local video code stream that local video code stream set includes to be sent to pairThe meeting-place terminal answered.
When entering to rack meeting, each meeting-place terminal is required to the local video code stream of oneself being sent to serviceDevice.Now, server no longer carries out decoding conjunction to the local video code stream transmitted by each meeting-place terminalInto waiting processing, but for each meeting-place terminal, respectively according to default rule determine to its issue it is localVideo code flow set, and the set of local video code stream at least includes a local video code stream；Then further according toThe local video code stream set issued to meeting-place terminal determined, the sheet that the local video code stream set is includedGround video code flow is sent to each meeting-place terminal, can effectively save the resource of server, allows each meeting-placeThe processing of terminal-pair local video code stream is shared.It is that each meeting-place is whole that server, which sends local video code stream,What end was sent respectively, it there will not be interference between each meeting-place terminal.Due to the meeting-place terminal of participant and more thanOne, therefore, the local video code stream of meeting-place terminal also has a lot；So, the first sending module 102In difference implication, be exactly that local video code stream needed for each meeting-place terminal is likely to more than one, thatJust the local video code stream needed for the meeting-place terminal is forwarded respectively；Server by receive eachThe local video code stream of meeting-place terminal, untreated direct forwarding, is transmitted to the required of each meeting-place terminalLocal video code stream be original local video code flow that each meeting-place terminal is sent to server.Local videoCode stream set, refers to that each terminal reports the set of the local video code stream of server, and this set is formerThe set of the local video code stream of beginning, and wrapped in local video code stream set corresponding to each meeting-place terminalThe local video code stream included is probably identical, it is also possible to part identical, or different.
Above-mentioned default rule, can include：By the local video code stream of broadcast source, local video is used asA part in code stream set, the i.e. set of local video code stream include the local video code stream of broadcast source；May be used alsoWith including receiving the request of meeting-place terminal, it will the local video code stream of field terminal request is used as local videoA part in code stream set, the i.e. set of local video code stream include the local video code stream of meeting-place terminal request.
In general, the local video code stream for being sent to some meeting-place terminal should be different from meeting-place endThe local video code stream at end, certainly, the meeting-place terminal local video code stream of oneself are need not to be connect from serverReceive；In cloud meeting, any meeting-place terminal should at least have the picture of a meeting-place terminal, when moreTime can have multiple pictures.Because the meeting-place terminal number of participant is not unique, and each meeting-place terminalIt is not to be required for seeing all meeting-place terminals, some meeting-place terminals, may for the consideration given top priority to what is the most importantThe video of some meeting-place terminal can be only checked, some meeting-place terminals again perhaps to look at that other meeting-place are whole moreThe picture at end and select to check the video of all meeting-place terminals, therefore, server what is desired is that, according to notThe demand of same meeting-place terminal, the local video code stream for those meeting-place terminals that these meeting-place terminals are needed is sent outGive these meeting-place terminals.
In cloud meeting, broadcast source is usually had, broadcast source is also one in the terminal of meeting-place, i.e., whole cloudThe overall situation of meeting is controlled by this meeting-place terminal, therefore, generally requires to regard the local of this meeting-place terminalFrequency code stream is sent to each meeting-place terminal, then, now, the set of local video code stream includes broadcast sourceLocal video code stream, the first sending module 102 includes video sending module 1021, for by the sheet of broadcast sourceGround video code flow is sent to other each meeting-place terminals；In addition, broadcast source is also required for what is pointed in a meetingSome or some meeting-place terminals are seen, it is corresponding, it is necessary to which the local video code stream of these meeting-place terminals is sentTo broadcast source；Broadcast source can see other meeting-place terminals, can also see broadcast source oneself, broadcast source is seenMeeting-place terminal, be known as the end that broadcast source is seen；In a cloud meeting, the two are often had more specialDifferent meeting-place terminal, wherein, the local video code stream of broadcast source needs to be sent to each meeting-place terminal, and wideThe end that the source of broadcasting is seen needs to be sent to broadcast source.Certainly, the end that broadcast source is seen is to become in a cloud meetingChange, and may be regular change, this has no effect on the implementation of this programme.In addition, for eachReason is planted, broadcast source is also to change in once cloud meeting in itself, after broadcast source change,The local video code stream of broadcast source after change should be sent to each meeting-place terminal.
In addition, determining module 105 also includes request receiving module 1051；Each meeting-place terminal is in cloud meetingWhenever, whether just start, or in a meeting, can send and ask to server, thisRequest at least includes the meeting-place terminal that meeting-place terminal needs to see, in other words, and each meeting-place terminal is to serviceDevice, which is sent, includes the request for the meeting-place terminal that needs are seen；Request receiving module 1051 is then used to receive the request,Determine that the local video code stream needed for the meeting-place terminal, the i.e. set of local video code stream include meeting-place terminal requestLocal video code stream, video sending module 1021 then according to the request to meeting-place terminal send it is corresponding thisGround video code flow.In other words, each meeting-place terminal should at least receive the local video code stream of broadcast source,Further, it is also possible to receive the local video code stream of other meeting-place terminals.Video sending module 1021 be additionally operable to byThe local video code stream of meeting-place terminal request is sent to meeting-place terminal.
Optionally, in addition to the first audio receiving module 103 and audio processing modules 104；Server is being receivedDuring the local video code stream of each meeting-place terminal, accordingly, each is received by the first audio receiving module 103First audio code stream of meeting-place terminal；When receiving the first audio code stream of each meeting-place terminal, really, clothesFirst audio code stream of each meeting-place terminal directly can also be sent to each meeting-place terminal and be handled by business device,However, due to audio processing with video processing compared to its occupancy server CPU and internal memory it is relatively low,Therefore it is not necessarily to；In addition, proceed from the reality, due to each with can meeting-place terminal in a meeting and be not required toSee the picture of each meeting-place terminal, but generally require the sound for hearing all meeting-place terminals, soThe meeting that can enable is normally carried out.If meeting-place terminal selectively is received to the first audio code stream stream,The meeting-place terminal can be then caused to miss the speech of other some meeting-place terminals, if the speech missed is especially importantWords, then can directly affect the quality of this meeting.Therefore, transmitted by each meeting-place terminal in the present embodimentThe first audio code stream, handled, then sent by 104 pairs of first audio code streams of audio processing modulesGive each meeting-place terminal.Audio processing modules 104 include：Audio decoder module 1041, mix module 1042,Coding module 1043, the first audio sending module 1044；Wherein, audio decoder module 1041 is used for theOne audio code stream is decoded, and obtains at least one audio；Mix module 1042 is used to mix the audioSound；Coding module 1043 is used to be encoded the audio after audio mixing, obtains the second audio code stream；First soundFrequency sending module 1044 is used to the second audio code stream being sent to each meeting-place terminal.Wherein, to each meeting-placeThe first audio decoder and/or the second audio coding that terminal is sent can include Waveform Codec, parametric codecAnd any feasible mode in mixed encoding and decoding；Waveform Codec includes PCM, ADPCM, SB-ADPCMEtc. code encoding/decoding mode, the parametric codec code encoding/decoding mode such as including LPC, mixed encoding and decoding include CELPC,VSLPC, RPE-LTP, LD-CELP, MPE etc. code encoding/decoding mode.It is audio frequency process to carry out audio mixing to audioIn a step, be the sound a variety of sources, be integrated into a sound.Each sound in the present embodimentFrequency from different meeting-place terminals, in mix process, can by the frequency of each audio, dynamic, tonequality,Positioning, reverberation and sound field are individually adjusted, and are allowed each track to optimize, are superimposed on again in final finished afterwards.This processing mode, can produce well-bedded audio frequency effect.Audio mixing can be located by audio mixing softwareReason.Audio after audio mixing is encoded again, it is possible to be sent to each meeting-place terminal.Server can be rightAll audios are handled, but in some cases, what server can also be selective is whole to each meeting-placeThe audio code stream at end is handled, and this is probably that need not be spoken because of the meeting-place terminal, or by broadcast sourceAllocated, by the way of making a speech in order etc..
A kind of meeting-place terminal is present embodiments provided, Fig. 6 is refer to, including：
Second sending module 201, for the local video code stream of meeting-place terminal to be sent into server；
Second receiving module 202, is locally regarded for what the local video code stream set that the reception server is issued includedFrequency code stream；
Video processing module 203, for handling local video code stream, obtains video.
In cloud meeting, each meeting-place terminal for adding cloud meeting should be by respective local video code streamIt is sent to server.Although each Cloud Server is in same cloud meeting, each meeting-place terminal is sentThe process of local video code stream is independent, i.e., each meeting-place terminal is respectively by respective local video code streamServer is sent to, likewise, at least one other meeting-place terminal of each meeting-place terminal the reception serverThe process of local video code stream is also independent, will not be interfered with each other between each meeting-place terminal.
The local video code stream that the local video code stream set that the reception server is issued includes is referred in cloud meetingIn view, server have received the local video code stream that each meeting-place terminal is sent respectively, then by local videoCode stream is sent respectively to each meeting-place terminal；Each meeting-place terminal local video code stream to be received should beThe local video code stream of other meeting-place terminals, and the local video code stream of meeting-place terminal oneself need not be received；ThisThe local video code stream that ground video code flow set includes, it is however generally that, refer in cloud meeting, any meeting-placeTerminal should at least have the picture of a meeting-place terminal, there is multiple pictures when more.Due to the meeting of participantField terminal number is not unique, and each meeting-place terminal is not to be required for seeing all meeting-place terminals,Some meeting-place terminals only may can check the video of some meeting-place terminal, have for the consideration given top priority to what is the most importantMeeting-place terminal again perhaps to looking at the picture of other meeting-place terminals more and selecting to check all meeting-place terminalsVideo, therefore, server what is desired is that, it is according to the demand of different meeting-place terminals, these meeting-place are wholeThe local video code stream of those the meeting-place terminals needed is held to be sent to these meeting-place terminals.Local video code stream collectionClose, refer to that each terminal reports the set of the local video code stream of server, this set is original sheetThe set of ground video code flow, and sheet included in local video code stream set corresponding to each meeting-place terminalGround video code flow is probably identical, it is also possible to part identical, or different.
In cloud meeting, broadcast source is usually had, broadcast source is also one in the terminal of meeting-place, i.e., whole cloudThe overall situation of meeting is controlled by this meeting-place terminal, therefore, generally requires to regard the local of this meeting-place terminalFrequency code stream is sent to each meeting-place terminal, then, the second receiving module 202 includes video reception module 2021,Local video code stream for receiving broadcast source, i.e. local video code stream set includes the local video of broadcast sourceCode stream；Certainly, here receive or as transmitted by server；In addition, broadcast source is also needed in a meetingHave sensing sees some or some meeting-place terminals, corresponding, it is necessary to which the local of these meeting-place terminals is regardedFrequency code stream is sent to broadcast source, that is to say, that broadcast source needs to receive the local video code stream at the end that broadcast source is seen；Broadcast source can see other meeting-place terminals, can also see broadcast source oneself, the meeting-place terminal that broadcast source is seen,It is known as the end that broadcast source is seen；In a cloud meeting, the two more special meeting-place terminals are often had,Wherein, the local video code stream of broadcast source needs to be sent to each meeting-place terminal, and the end needs that broadcast source is seenIt is sent to broadcast source.Certainly, the end that broadcast source is seen can change in a cloud meeting, and mayThe regular change of meeting, this has no effect on the implementation of this programme.In addition, for various reasons, broadcast sourceItself it is also to change in once cloud meeting, also should be by after change after broadcast source changeThe local video code stream of broadcast source is sent to each meeting-place terminal.
In addition, the second receiving module 202 also includes request sending module 2022；Please for being sent to serverAsk；Each meeting-place terminal in cloud meeting whenever, whether just start, or in a meeting, allIt can send and ask to server, this request at least includes the meeting-place terminal that the meeting-place terminal needs to see,I other words, each meeting-place terminal to server, which is sent, includes the request for the meeting-place terminal that needs are seen；Server is thenCorresponding local video code stream is sent to the meeting-place terminal according to the request.The set of local video code stream includes meetingThe local video code stream of field terminal request.In other words, each meeting-place terminal should at least receive broadcast sourceLocal video code stream, further, it is also possible to receive the local video code stream of other meeting-place terminals.Video reception mouldBlock 2021 is additionally operable to receive the local video code stream of request, i.e., the meeting-place that request sending module 2022 is asked is wholeThe local video code stream at end.
After at least one local video code stream is received, it is possible to which local video code stream is handled；Depending onFrequency processing module 203 includes Video decoding module 2031 and synthesis module 2032, Video decoding module 2031For being decoded to local video code stream, at least one sub-video is obtained；Then the antithetical phrase of synthesis module 2032Video performs synthetic operation, obtains video.The local video code stream that meeting-place terminal is received, including broadcast source existIt is interior, may have multiple, terminal-pair this multiple local video code stream in meeting-place are decoded；The encoding and decoding of videoForm mainly has following several：H.261, H.263, H.264, using any form.By eachAfter local video code stream decoding, at least one sub-video has been obtained, it is possible to according to the meeting-place terminal meaning of oneselfHope is synthesized to each sub-video.The screen layout of video after synthesis can be arbitrary, and some are commonly usedSynthesis after screen layout can be：First, corresponding to the sub-video of each meeting-place terminal according to same bigIt is small to be uniformly distributed in picture；Second, using one of sub-video as main video, make its picture maximum, itsHis sub-video is as from video, and picture is distributed in around key frame, wherein, main view frequency can be with broadcastAs main view frequency, or in now cloud meeting, the video of the meeting-place terminal of main presentation is used as main view frequency in source；ItsThree, the picture of the video after synthesis can be dynamic, the corresponding amplification if whose video pictures who makes a speech,Seem than more prominent.Using any of the above mode or other NM synthesis modes in the present embodimentIt is applicable, as long as its user that the video in the meeting-place terminal can be allowed normally to be shown to participant sees.So, the processing procedure of the decoding synthesis of local video code stream is gone to corresponding meeting-place terminal to handle, byBe in the processing of each meeting-place terminal it is independent, not only can be customized according to each meeting-place terminal individual, have veryGood Consumer's Experience, the CPU and internal memory for also reducing server end is used so that server can be accessed moreMany meeting-place terminals, so as to improve the efficiency of meeting.
In addition, also including the second audio sending module 204, for sending the local video code stream of meeting-place terminalWhile to server, the first audio code stream of each meeting-place terminal is sent to server.Sending eachDuring the first audio code stream of meeting-place terminal, really, meeting-place terminal can also be using similar local video code streamProcessing mode is handled audio code stream, but audio code stream has its particularity, the processing of audio and videoProcessing compare, the CPU and internal memory of the server shared by audio frequency process are relatively low, thus be not necessarily to；ThisOutside, proceed from the reality, due to each with can meeting-place terminal in a meeting and need not see that each meeting-place is wholeThe picture at end, but the sound for hearing all meeting-place terminals is generally required, it could so enable meeting justOften carry out.Therefore, in addition to the second audio receiving module 205, each meeting-place terminal sends out the first audio code streamGive after server, server is handled it, obtain the second audio code stream, then by the second audio codeStream is sent to each meeting-place terminal.Server to consistent in the processing and above example of the first audio code stream,Here repeat no more.
Above content is to combine specific embodiment further description made for the present invention, it is impossible to recognizedThe specific implementation of the fixed present invention is confined to these explanations.For the ordinary skill of the technical field of the inventionFor personnel, without departing from the inventive concept of the premise, some simple deduction or replace can also be made,Protection scope of the present invention should be all considered as belonging to.