CN105451100A - Method and device for playing video streams - Google Patents

Abstract

The invention discloses a method and a device for playing video streams. The method comprises the following steps: receiving video frames, calculating a maximum value in network delays of the received video frames, and putting the received video frames into a video frame buffer queue; and reading an earliest video frame from the video frame buffer queue according to time stamps of the video frames, calculating playing time for the read video frame, and removing the read video frame from the video frame buffer queue and performing decoding and playing if current time is greater than or equal to the playing time of the read video frame, wherein the playing time= (time stamp time of the video frame) + (the maximum value in the network delays of the received video frames). Through adoption of the method and the device, sequential playing of the video frames can be ensured under the condition of ensuring that a quantity of buffered video frames is as small as possible; the video frame playing continuity is ensured; and the timeliness of video frame playing is ensured.

Description

A kind of method of displaying video stream and device

Technical field

The present invention relates to stream media technology, espespecially a kind of method of displaying video stream and device.

Background technology

Stream media technology (streamingmediatechnology) be a kind of can make the multimedia such as audio frequency, video on internet and intranet with real-time, carry out without the need to downloading the mode waited for the technology play.Video flowing (videostream) is the video data play in stream media technology.Frame of video (videoframe) is the component units of video flowing, and a frame of video is exactly the data after a sub-picture digitlization, and video flowing is made up of continuous print frame of video one by one.

Stream media technology more and more receives an acclaim in multimedia technology field without the need to downloading the characteristic waited for because of it, plays more and more important effect in numerous occasion.Wherein, the buffering of video flowing and to play be an important technology in stream media technology field, the quality of its mechanism directly affects the perception performances such as instantaneity that video data plays and fluency.

Video stream data is rearranged in order by frame of video one by one, in the process through Internet Transmission, due to the unsteadiness of network, the order receiving frame of video may be different with the order produced when gathering, therefore, the terminal of receiver, video stream needs to sort to frame of video, and the order produced when then gathering according to frame of video is play.And the frame of video received is sorted just need to carry out buffer memory to frame of video, the number of the quantity of the frame of video of buffer memory will affect the instantaneity of broadcasting.And buffer memory should under the prerequisite that ensure playing sequence the try one's best few buffered video frame machine-processed with broadcasting that good.

Summary of the invention

In order to solve the problems of the technologies described above, the invention provides a kind of method and device of displaying video stream, when ensureing as far as possible few buffered video frame, the played in order of frame of video can be ensured.

In order to reach the object of the invention, the invention provides a kind of method of displaying video stream, comprising:

Receiver, video frame, calculates the maximum in the network delay of the frame of video received, the frame of video received is put into frame of video buffer queue;

Timestamp according to frame of video reads time frame of video the earliest from frame of video buffer queue, for the frame of video read out calculates reproduction time, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play; Wherein, described reproduction time=frame of video timestamp time+the described frame of video received network delay in maximum.

Further, the maximum in the network delay of the frame of video that described calculating has received, comprising:

According to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

Further, the described frame of video for reading out calculates reproduction time, comprising:

According to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates described reproduction time, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, according to delay_i-delay₁=(at_i-at₁)-(ts_i-ts₁) calculate described MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, described receiver, video frame, comprising:

If what receive is first frame of video, then record the timestamp time ts of first frame of video₁with the time at receiving first frame of video₁, and frame of video is put into frame of video buffer queue; If what receive is subsequent video frame, then according to delay_i-delay₁computing network time delay relative value, and recalculate the maximum MaxDelayDelta in the network delay relative value of all frame of video received, then frame of video is put into frame of video buffer queue.

Further, a kind of device of displaying video stream, comprising: receiving element, the first computing unit, broadcast unit and the second computing unit, wherein:

Described receiving element, for receiver, video frame, puts into frame of video buffer queue by the frame of video received;

Described first computing unit, for calculating the maximum in the network delay of the frame of video that described receiving element has received;

Described broadcast unit, for reading time frame of video the earliest according to the timestamp of frame of video from frame of video buffer queue, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play;

Described second computing unit, calculates reproduction time for the frame of video that reads out for described broadcast unit, wherein, the timestamp of described reproduction time=frame of video time+the described frame of video received network delay in maximum.

Further, described first computing unit calculates the maximum in the network delay of the frame of video received, and comprising:

According to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

Further, described second computing unit is that the frame of video that described broadcast unit reads out calculates reproduction time, comprising:

According to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates described reproduction time, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, described first computing unit, also for according to delay_i-delay₁=(at_i-at₁)-(ts_i-ts₁) calculate described MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, described receiving element receiver, video frame, comprising:

If what receive is first frame of video, then record the timestamp time ts of first frame of video₁with the time at receiving first frame of video₁, and frame of video is put into frame of video buffer queue; If what receive is subsequent video frame, then indicate described first computing unit according to delay_i-delay₁computing network time delay relative value, and recalculate the maximum MaxDelayDelta in the network delay relative value of all frame of video received, then frame of video is put into frame of video buffer queue.

Compared with prior art, the present invention includes receiver, video frame, calculate the maximum in the network delay of the frame of video received, the frame of video received is put into frame of video buffer queue; Timestamp according to frame of video reads time frame of video the earliest from frame of video buffer queue, for the frame of video read out calculates reproduction time, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play; Wherein, described reproduction time=frame of video timestamp time+the described frame of video received network delay in maximum.The present invention when ensureing the quantity of as far as possible few buffered video frame, can ensure again the played in order of frame of video simultaneously, both having ensured that presentation of video frames continuity in turn ensure that the instantaneity of presentation of video frames.

Other features and advantages of the present invention will be set forth in the following description, and, partly become apparent from specification, or understand by implementing the present invention.Object of the present invention and other advantages realize by structure specifically noted in specification, claims and accompanying drawing and obtain.

Accompanying drawing explanation

Accompanying drawing is used to provide the further understanding to technical solution of the present invention, and forms a part for specification, is used from and explains technical scheme of the present invention, do not form the restriction to technical solution of the present invention with the embodiment one of the application.

Fig. 1 is the flow chart of the method for displaying video stream of the present invention;

Fig. 2 is the Organization Chart of the device of displaying video stream of the present invention.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly understand, hereinafter will be described in detail to embodiments of the invention by reference to the accompanying drawings.It should be noted that, when not conflicting, the embodiment in the application and the feature in embodiment can combination in any mutually.

Can perform in the computer system of such as one group of computer executable instructions in the step shown in the flow chart of accompanying drawing.Further, although show logical order in flow charts, in some cases, can be different from the step shown or described by order execution herein.

As shown in Figure 1, the method for displaying video stream of the present invention, comprising:

Step 101: receiver, video frame, calculates the maximum in the network delay of the frame of video received, the frame of video received is put into frame of video buffer queue;

The present invention calculates the maximum in the network delay of the frame of video received, and comprising: according to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

Suppose that the timestamp time of first frame of video (frame of video 1) when gathering received is ts₁, the timestamp time of frame of video 2 when gathering is ts₂, the time that frame of video 1 arrives receiving terminal is at₁(also namely receiving the time of first frame of video), the time that frame of video 2 arrives receiving terminal is at₂.

The network delay delay of frame of video 1₁be: delay₁=at₁-ts₁;

The network delay delay of frame of video 2₂be: delay₂=at₂-ts₂;

By that analogy, the network delay of frame of video n is: delay_n=at_n-ts_n.

The maximum calculated in the network delay of the frame of video received can, after the network delay calculating each frame of video received, adopt the method compared to obtain the maximum of network delay.

The present invention also can by calculating each network delay relative to delay₁the maximum of difference, then adding delay₁the network delay of frame of video that received of method in maximum.That is:

Max{delay₁,delay₂,...,delay_n}＝delay₁+Max{delay₁-delay₁,delay₂-delay₁,...,delay_n-delay₁}。

Step 102: the timestamp according to frame of video reads time frame of video the earliest from frame of video buffer queue, for the frame of video read out calculates reproduction time; Wherein, reproduction time=frame of video timestamp time+network delay of frame of video that received in maximum;

Frame of video generally understands affix acquisition time when gathering, and acquisition time is called timestamp, and the time sequencing that each frame of video gathers and interval can be known by timestamp in one end of receiver, video stream.Receiving terminal can obtain the time of each frame of video in transmission over networks, i.e. network delay by the difference calculating time when receiving each frame of video and acquisition time.Due to the unsteadiness of network, the network delay of each frame of video is generally different.By calculating, the maximum in the network delay of each frame of video can be obtained, this maximum delay adds that the acquisition time of frame of video is necessarily greater than or equal to the time that frame of video arrives receiving terminal, using the timestamp time of each frame of video, receiving terminal adds that the maximum of network delay just can ensure before reproduction time reaches, necessarily have received the frame that will play as the reproduction time of frame of video, also just ensure that the continuity that frame is play.

The timestamp time of the frame of video that can just read in the present invention is directly added with the maximum in the network delay of the frame of video calculated the reproduction time obtaining the frame of video read.

When calculating the maximum in the network delay of frame of video received, if adopt Max{delay₁, delay₂..., delay_n}=delay₁+ Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, also can according to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates the reproduction time of frame of video read, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

If Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁}=MaxDelayDelta, so the maximum MaxDelay of network delay is:

MaxDelay=delay₁the reproduction time rt of+MaxDelayDelta, frame of video i_ibe then:

rt_i＝ts_i+delay₁+MaxDelayDelta＝ts_i+(at₁-ts₁)+MaxDelayDelta

＝at₁+(ts_i-ts₁)+MaxDelayDelta。

Reproduction time can be calculated without the need to receiving terminal and collection terminal clock synchronous by the method.

Further, the present invention can according to delay_i-delay₁=(at_i-at₁)-(ts_i-ts₁) calculate MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

The network delay of frame of video i (1≤i≤n) is relative to delay₁difference be:

Delay_i-delay₁=(at_i-ts_i)-(at₁-ts₁)=(at_i-at₁)-(ts_i-ts₁), that is:

delay_i-delay₁＝(at_i-at₁)-(ts_i-ts₁)。

By this conversion, avoid the receiving terminal clock problem that cause calculating inaccurate asynchronous with collection terminal clock.

Step 103: if current time is more than or equal to the reproduction time of the frame of video read out, then remove the frame of video read out and carry out decoding and play from frame of video buffer queue.

Below with according to rt_i=at₁+ (ts_i-ts₁) to calculate the reproduction time of frame of video read be that example further illustrates method of the present invention to+MaxDelayDelta.

Step a: frame of video receiving thread is from network reception frame of video, if first time receives frame of video just record its timestamp ts₁with the time at arrived₁, and put it into frame of video buffer queue; Frame of video is received, then computing network time delay relative value delay if not first time_i-delay₁, and recalculate the maximum MaxDelayDelta of the network delay relative value of all frames received, finally put it into frame of video buffer queue;

The size of temporally being stabbed by frame of video i is inserted into frame of video buffer queue, ensures that the size that queue is temporally stabbed arranges.

Step b: presentation of video frames thread reads the oldest frame of video of timestamp, according to rt from frame of video buffer queue_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates its reproduction time;

Step c: if current time is more than or equal to the reproduction time of the frame of video of reading, removes this frame of video and carries out decoding and play, otherwise continue to wait for from queue.

As shown in Figure 2, present invention also offers a kind of device of displaying video stream, comprising: receiving element, the first computing unit, broadcast unit and the second computing unit, wherein:

Receiving element, for receiver, video frame, puts into frame of video buffer queue by the frame of video received;

First computing unit, for calculating the maximum in the network delay of the frame of video that receiving element has received;

Broadcast unit, for reading time frame of video the earliest according to the timestamp of frame of video from frame of video buffer queue, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play;

Second computing unit, calculates reproduction time for the frame of video that reads out for broadcast unit, wherein, the timestamp of reproduction time=frame of video time+network delay of frame of video that received in maximum.

Further, the first computing unit calculates the maximum in the network delay of the frame of video received, and comprising:

According to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

Further, the second computing unit is that the frame of video that broadcast unit reads out calculates reproduction time, comprising:

According to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates reproduction time, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, the first computing unit, also for according to delay_i-delay₁=(at_i-at₁)-(ts_i-ts1) calculate MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

Further, receiving element receiver, video frame, comprising:

If what receive is first frame of video, then record the timestamp time ts of first frame of video₁with the time at receiving first frame of video₁, and frame of video is put into frame of video buffer queue; If what receive is subsequent video frame, then indicate the first computing unit according to delay_i-delay₁computing network time delay relative value, and recalculate the maximum MaxDelayDelta in the network delay relative value of all frame of video received, then frame of video is put into frame of video buffer queue.

Although the execution mode disclosed by the present invention is as above, the execution mode that described content only adopts for ease of understanding the present invention, and be not used to limit the present invention.Those of skill in the art belonging to any the present invention; under the prerequisite not departing from the spirit and scope disclosed by the present invention; any amendment and change can be carried out in the form implemented and details; but scope of patent protection of the present invention, the scope that still must define with appending claims is as the criterion.

Claims (10)

1. a method for displaying video stream, is characterized in that, comprising:

Receiver, video frame, calculates the maximum in the network delay of the frame of video received, the frame of video received is put into frame of video buffer queue;

Timestamp according to frame of video reads time frame of video the earliest from frame of video buffer queue, for the frame of video read out calculates reproduction time, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play; Wherein, described reproduction time=frame of video timestamp time+the described frame of video received network delay in maximum.

2. method according to claim 1, is characterized in that, the maximum in the network delay of the frame of video that described calculating has received, comprising:

According to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

3. method according to claim 2, is characterized in that, the described frame of video for reading out calculates reproduction time, comprising:

According to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates described reproduction time, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

4. according to the method in claim 2 or 3, it is characterized in that:

According to delay_i-delay₁=(at_i-at₁)-(ts_i-ts₁) calculate described MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

5. method according to claim 4, is characterized in that, described receiver, video frame, comprising:

If what receive is first frame of video, then record the timestamp time ts of first frame of video₁with the time at receiving first frame of video₁, and frame of video is put into frame of video buffer queue; If what receive is subsequent video frame, then according to delay_i-delay₁computing network time delay relative value, and recalculate the maximum MaxDelayDelta in the network delay relative value of all frame of video received, then frame of video is put into frame of video buffer queue.

6. a device for displaying video stream, is characterized in that, comprising: receiving element, the first computing unit, broadcast unit and the second computing unit, wherein:

Described receiving element, for receiver, video frame, puts into frame of video buffer queue by the frame of video received;

Described first computing unit, for calculating the maximum in the network delay of the frame of video that described receiving element has received;

Described broadcast unit, for reading time frame of video the earliest according to the timestamp of frame of video from frame of video buffer queue, if current time is more than or equal to the reproduction time of the frame of video read out, then the frame of video read out is removed from frame of video buffer queue and carry out decoding and play;

Described second computing unit, calculates reproduction time for the frame of video that reads out for described broadcast unit, wherein, the timestamp of described reproduction time=frame of video time+the described frame of video received network delay in maximum.

7. device according to claim 6, is characterized in that, described first computing unit calculates the maximum in the network delay of the frame of video received, and comprising:

According to MaxDelay=delay₁+ MaxDelayDelta calculates the maximum in the network delay of the frame of video received, and wherein, MaxDelay is the maximum in the network delay of the frame of video received, and MaxDelayDelta is Max{delay₁-delay₁, delay₂-delay₁..., delay_n-delay₁, n>=1, delay₁for the network delay of first frame of video received, delay_nfor the network delay of the n-th frame of video received.

8. device according to claim 7, is characterized in that, described second computing unit is that the frame of video that described broadcast unit reads out calculates reproduction time, comprising:

According to rt_i=at₁+ (ts_i-ts₁)+MaxDelayDelta calculates described reproduction time, wherein, 1≤i≤n, rt_ifor the reproduction time of i-th frame of video received, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

9. the device according to claim 7 or 8, is characterized in that:

Described first computing unit, also for according to delay_i-delay₁=(at_i-at₁)-(ts_i-ts₁) calculate described MaxDelayDelta, wherein, 1≤i≤n, delay_ifor the network delay of i-th frame of video received, at_ifor receiving the time of i-th frame of video, at₁for receiving the time of first frame of video, ts_ifor the timestamp time of i-th frame of video received, ts₁for the timestamp time of first frame of video received.

10. device according to claim 9, is characterized in that, described receiving element receiver, video frame, comprising:

If what receive is first frame of video, then record the timestamp time ts of first frame of video₁with the time at receiving first frame of video₁, and frame of video is put into frame of video buffer queue; If what receive is subsequent video frame, then indicate described first computing unit according to delay_i-delay₁computing network time delay relative value, and recalculate the maximum MaxDelayDelta in the network delay relative value of all frame of video received, then frame of video is put into frame of video buffer queue.