Summary of the invention
The application provides a kind of processing method of video traffic, effectively can control buffer and export to the data of decoder, improve video playing quality.
For achieving the above object, the application adopts following technical scheme:
A data processing method in video traffic, comprising:
The video RTP received wraps and sends into video display buffer by receiving terminal; The video RTP of buffer memory bag sends to decoder to process by described video display buffer; Wherein, the result belonging to the RTP bag of same frame of video of buffer memory is sent to described decoder simultaneously;
The mode that the video RTP that described video display buffer sends buffer memory wraps comprises:
When the total amount N of bag data cached in video display buffer is lower than reference capacity thresholding Thred0, for initial data packets, before the initial buffer memory waiting time that the cache-time of data cached bag does not exceed setting, do not send data cached bag, after exceeding described initial buffer memory waiting time, receive the packet of the incoming frame frame per second transmission buffer memory of data according to described video display buffer; For non-initial packet, according to described incoming frame frame per second, send data cached bag; Wherein, described incoming frame frame per second is the quantity of receiver, video frame in the described video display buffer unit interval;
As the capacity threshold Thred1 of Thred0<N≤setting, if RTP packet delay shake average Jitteraveragethe delay variation thresholding of≤setting, then send described data cached bag according to the interval of timestamps of incoming frame; If the RTP packet delay shake average Jitter in timing statisticsaveragedelay variation thresholding described in >, then when described incoming frame frame per second is more than or equal to the Human Perception frame per second of setting, described data cached bag is sent according to the interval of timestamps of incoming frame, when described incoming frame frame per second is less than the Human Perception frame per second of setting, send described data cached bag according to described Human Perception frame per second; Wherein, the interval of timestamps of described incoming frame is that transmitting time between two frame of video that timestamp that described video display buffer receives is adjacent is poor;
As the capacity threshold Thred2 of Thred1<N≤setting, send described data cached bag according to the minimum value in the interval of timestamps of 1/ incoming frame frame per second and incoming frame;
When N>Thred2 or video display buffer overflow, the packet of the buffer memory more than Thred2 is all sent.
Preferably, the mode calculating described incoming frame frame per second comprises:
The number of receiver, video frame in the timing statistics of setting, by the frame of video of reception divided by described setting-up time; Or, determine that described buffer receives the time required for frame of video of setting number, by described setting number divided by the time determined.
Preferably, when calculating RTP packet delay shake average, in the timing statistics of setting, calculation delay shake initial mean value, on the basis of at least twice delay variation initial mean value, carries out long-term filtering and determines described delay variation average.
Preferably, the method comprises further: the audio frequency RTP received wraps and sends into audio buffer by receiving terminal, and audio buffer audio frequency RTP according to the transmission frequency synchronized transmission of described video display buffer wraps and processes to decoder.
As seen from the above technical solution, in the application, when the data volume in buffer is less, can ensure to retain certain data volume in buffer memory, thus when avoiding time delay to increase, the continuity of presentation of video frames is deteriorated; When the data volume in buffer is medium, ensure that delay variation is in the limit that can tolerate, ensures certain buffer data size on the other hand on the one hand; When the data volume in buffer is more, certain clearance spaces can be ensured as far as possible, reduce overflow probability.Visible, the mode in the application, effectively can control buffer and export to the data of decoder, improve video playing quality.
Embodiment
In order to make the object of the application, technological means and advantage clearly understand, below in conjunction with accompanying drawing, the application is described in further details.
In the application, as shown in Figure 1, buffer, between RTP layer and decoder layer, is responsible for buffer memory RTP packet in the position of buffer, and goes out bag control strategy according to certain RTP packet is issued decoder process.Spatial cache size can static state setting fixed value, or according to physical device ability dynamic assignment.RTP packet for audio frequency, video can be divided into two buffers to carry out buffer memory, and in the same time, amount of audio data can be significantly less than the video data volume, and the application mainly describes buffer memory for video data and transmission.For synchronous voice data, Frequency Synchronization can be sent according to video data simultaneously send corresponding voice data.Synchronization policy the present invention do not retrain.The most basically, when carrying out buffer memory for the RTP bag received, based on No. SN sequence of carrying in RTP bag.Need export according to the order of sequence according to No. SN equally when exporting from buffer memory.Belong to entrained timestamp timestamp value in the RTP bag of same frame of video identical; The RTP bag belonging to same frame of video sends simultaneously.
Provide the concrete control mode that buffer wraps to decoder output video RTP below.
When carrying out bag and controlling, first arrange three capacity threshold, as shown in Figure 2, buffer carries out packet when exporting, and specifically controls according to the relation between three capacity threshold of current cache data volume and setting.Three capacity threshold are respectively Thred0, Thred1 and Thred2, Thred0<=Thred1<=Thred2LEssT.LT ssT.LT=100%; Wherein, (100%-capacity threshold Thred2) * buffer size <=decoder processes cache size, avoids producing decoder processes when transient data amount is larger impacting.
Based on above-mentioned three capacity threshold, the control treatment that buffer carries out video RTP bag is as follows:
If 1 buffer memory is lower than capacity threshold Thred0, give out a contract for a project by following
If-primary data sends, the data cached bag in buffer waits for that time delay does not exceed " the initial buffer memory waiting time " of setting, wouldn't outwards give out a contract for a project; Data cached bag in buffer according to incoming frame frame per second, carries out frame of video transmission after waiting for that time delay reaches " initial buffer memory waiting time ".Like this, data volume certain in buffer can be ensured, avoid video content when initial play, can not be excessive due to time delay between packet, cause video playback discontinuous; Meanwhile, when also avoiding video initial play, period of reservation of number is long, improves Consumer's Experience;
If not-primary data sends, according to incoming frame frame per second, carry out frame of video transmission.
If 2 Thred1> buffer memory >=Thred0, dynamically give out a contract for a project by following:
" delay variation thresholding " that the delay variation average <=that-Ruo wraps calculating according to the video RTP of buffer memory presets, then timestamp (timestamp) interval based on incoming frame sends.That is, when current delay variation situation is in the limit tolerated, carries out bag according to the transmission frequency of frame of video and control.Here, the interval of timestamps of incoming frame refers to that the transmitting time between two frame of video that timestamp that buffer receives is adjacent is poor.
-Ruo delay variation average > " shake thresholding ", judges the relation of incoming frame frame per second and Human Perception frame per second further, if incoming frame frame per second >=Human Perception frame per second, then sends according to the timestamp interval of incoming frame; If incoming frame frame per second < Human Perception frame per second, then carry out frame transmission based on Human Perception frame per second.Here, human eye has a minimum perception frame per second for dynamic video, is called Human Perception frame per second, and when video frame rate is lower than this Human Perception frame per second, the video fluency that Human Perception arrives is deteriorated.Can according to practical application scene settings Human Perception frame per second.This situation shows, when current delay variation situation is not in the limit tolerated, buffer is minimum to be exported with Human Perception frame per second.Like this, ensure the fluency of video playback on the one hand, ensure that the data volume of buffer memory can not be too much on the other hand.
From the process of two kinds of situations in above-mentioned 1 and 2, arranging of Thred0 needs to ensure certain data volume on the one hand, when time delay becomes large, when causing subsequent data to arrive continuity variation, still can ensure that continuous print data send by buffer memory; Need on the other hand to consider user's maximum wait duration; Comprehensively above-mentioned 2 settings carrying out Thred0, such as, can be set to 20%.
If 3 Thred2 >=buffer memory >=Thred1, dynamically give out a contract for a project by following:
The timestamp interval of (1/ incoming frame frame per second) < incoming frame that-Ruo is current, then carry out data transmission according to incoming frame frame per second;
The timestamp interval of (1/ incoming frame frame per second) >=incoming frame that-Ruo is current, then carry out data transmission according to the timestamp interval of incoming frame.
When buffer memory meets this situation, need to limit the buffer data size in buffer, therefore the maximum sent in the frame per second of frame of video according to frame per second and the transmitting terminal of buffer receiver, video frame exports, thus the buffer data size controlled in buffer does not raise too much as far as possible.
From above-mentioned 2 and the process of 3 two kind of situation, arranging of Thred1 will ensure that buffer memory does not have pressure on the one hand, will consider the fluctuation entering frame rate that can tolerate on the other hand.Comprehensively above-mentioned 2 settings carrying out Thred1, such as, can be set to 50%.
If 4 buffer memory >Thred2 or instantaneous reception data cause cache overflow, then the frame data more than Thred2 are all sent.Here process, when ensureing that buffer data size is more in both the buffers, sends packet as early as possible, overflows to avoid buffer.
From above-mentioned 3 and the process of 4 two kind of situation, after buffer memory is higher than Thred2, show that buffer memory accounting is higher, at this moment remaining spatial accommodation diminishes, and when transient data amount is higher, easily causes cache overflow, therefore, ensure certain clearance spaces as far as possible, reduce overflow probability.Therefore, need to arrange Thred2 on this basis, such as, can be set to 80%.
Go out to wrap in control above-mentioned, relate to the incoming frame frame per second of buffer and the delay variation average of RTP bag, introduce the concrete account forms of these two amounts below:
1) the incoming frame frame per second of video display buffer
The incoming frame frame per second that video cache buffer receives data is the number of video frames received in the unit interval.Can adopt following appraisal procedure, following method can independently use:
-be reference with set time, namely add up the number of video frames received in the fixed time, calculate incoming frame frame per second.
-with designated frame number for reference, namely statistics specifies the time receiving certain frame of video and need to consume, and calculates incoming frame frame per second.
Incoming frame frame per second is calculated as follows:
Frame number/the receiving terminal of incoming frame frame per second=receive receives the time (unit: frame/s) that this framing consumes
2) receiving terminal RTP packet delay shake average
The delay variation statistical of standard is defined in RTP/RTCP agreement.Namely the application adopts this statistical, repeats no more here.
Need in the application continue statistics receiving package delay variation, above-mentioned calculating incoming frame frame per second fixed time or specify frame number basis on, add up the delay variation average in this time interval, can above-mentioned go out bag control time directly utilize this delay variation average.Or, can also using the delay variation average that calculates in the manner described above its as delay variation initial mean value, between the statistical interval of at least twice, perform long-term filtering, filtering mode is not limit (such as alpha filtering), carries out the delay variation average that long-term level and smooth acquisition is final.
In addition, due to the difference of device own, may there is the accuracy that crystal oscillator difference causes two ends Time Calculation in transmitting terminal and receiving terminal.In the application, think very little compared with the change that difference that crystal oscillator introduces and propagation delay time are introduced, therefore, in the application, put aside the impact of crystal oscillator.If the two difference is excessive, need to consider that crystal oscillator compensates, and when the interval of timestamps according to incoming frame carries out the transmission of data cached bag, consider that crystal oscillator compensates in advance.The application does not retrain crystal oscillator compensation method.
By the process of above-mentioned the application, when the data volume in buffer is less, can ensure to retain certain data volume in buffer memory, thus when avoiding time delay to increase, the continuity of presentation of video frames is deteriorated; When the data volume in buffer is medium, ensure that delay variation is in the limit that can tolerate, ensures certain buffer data size on the other hand on the one hand; When the data volume in buffer is more, certain clearance spaces can be ensured as far as possible, reduce overflow probability.Visible, the mode in the application, effectively can control buffer and export to the data of decoder, improve video playing quality.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment made, equivalent replacement, improvement etc., all should be included within the scope of protection of the invention.