CN108260024B - Live broadcast data processing method and device, computer equipment and storage medium - Google Patents

Abstract

The application relates to a live data processing method, a system, a device, a computer readable storage medium and a computer device, wherein the method comprises the following steps: receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data; slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments; receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set; and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition. The scheme that this application provided can reduce the broadcast time error between the live broadcast end.

Description

Live broadcast data processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a live data processing method and apparatus, a computer-readable storage medium, and a computer device.

Background

With the development of computer technology, live webcasting is favored by users because of the advantages of images, sounds and characters, especially because of providing instant and interactive communication and communication modes for users, and in recent years, a new live webcasting interactive mode, such as live webcasting answering, is presented, and each large webcasting platform is online with its own product, but live webcasting in the new live webcasting interactive mode, such as live webcasting answering, needs to ensure the consistency of each live webcasting playing terminal well.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a live broadcast data processing method, a device, a computer-readable storage medium, and a computer device, which can reduce the error of the broadcast time between live broadcast terminals, thereby improving the consistency between live broadcast terminals.

A live data processing method includes:

receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data;

slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments;

receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set;

and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

A live data processing system, the system comprising:

the processing server is used for receiving the live streaming data, slicing the live streaming data according to a preset slicing condition and obtaining a plurality of corresponding live streaming fragment data;

the playing terminal is used for sending a downloading request to the processing server;

the processing server is also used for receiving a downloading request sent by the playing terminal, screening the plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set, and sending data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit;

the playing terminal is further used for detecting whether the received live broadcast fragments sent by the processing server meet preset playing conditions or not, and if the preset playing conditions are met, the received live broadcast fragments begin to be played.

A live data processing apparatus, the apparatus comprising:

the live streaming data receiving module is used for receiving live streaming data, and the live streaming data comprises live audio and video data and corresponding information data;

the live streaming data slicing module is used for slicing live streaming data according to preset slicing conditions to obtain a plurality of corresponding live streaming fragments;

a download request receiving module, configured to receive a download request sent by a play terminal;

the target live broadcast fragment screening module is used for screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set;

and the live broadcast fragment sending module is used for sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so as to enable the playing terminal to start playing the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps when executing the program of:

receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data;

slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments;

receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set;

and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of:

receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data;

slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments;

receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set;

and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

According to the live broadcast data processing method and device, the computer-readable storage medium and the computer equipment, the processing server encapsulates the live broadcast audio and video data and the corresponding information data into live broadcast stream data, and then slices the live broadcast stream data according to the preset slicing condition to obtain the plurality of live broadcast fragments, and the playing time of the live broadcast fragments can be controlled through the preset slicing condition, so that the playing time of the live broadcast fragments can be reduced. Further, when the playing terminal sends a download request to the processing server, the processing server screens the plurality of cached live broadcast fragments according to the download request to obtain a target live broadcast fragment set, and if a preset number of live broadcast fragments closest to a time point at which the download request is received are returned. And finally, sending the data in the target live broadcast fragment set to a playing terminal in each live broadcast fragment unit, wherein the playing terminal receives the live broadcast fragments in sequence, and playing the received live broadcast fragments when the received live broadcast fragments meet preset playing conditions. The processing server can control the playing time of the live broadcast fragments, and the playing terminal can also control the playing of the received live broadcast fragments, so that the playing time error between live broadcast playing terminals is reduced, and the consistency between the live broadcast playing terminals is improved.

Drawings

FIG. 1 is a diagram of an application environment of a live data processing method in one embodiment;

FIG. 2 is a flow diagram of a method for live data processing in one embodiment;

fig. 3 is a schematic flow chart of a live data processing method in another embodiment;

FIG. 4 is a flow diagram illustrating a live stream data slicing step in one embodiment;

FIG. 5 is a flowchart illustrating a step of generating a target live broadcast segment set in one embodiment;

fig. 6 is a schematic flow chart illustrating a step of playing a live broadcast segment by the playing terminal in one embodiment;

fig. 7 is a schematic diagram illustrating that the playing terminal plays the received live broadcast segment in one embodiment;

FIG. 8a is a diagram of an application scenario of a live data processing method in an embodiment;

FIG. 8b is a diagram of an application scenario of a live data processing method in another embodiment;

FIG. 8 is a timing diagram of a live data processing method in one embodiment;

FIG. 9 is a block diagram of a live data processing system in one embodiment;

fig. 10 is a block diagram of a live data processing system in another embodiment;

FIG. 11 is a block diagram of a live data processing apparatus in one embodiment;

fig. 12 is a block diagram showing the configuration of a live data processing apparatus according to another embodiment;

FIG. 13 is a block diagram that illustrates the structure of a live streaming data slicing module in one embodiment;

FIG. 14 is a block diagram that illustrates a block diagram that shows a target live segment filter module in one embodiment;

FIG. 15 is a block diagram showing a configuration of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Fig. 1 is an application environment diagram of a live data processing method in an embodiment. Referring to fig. 1, the live data processing method is applied to a live data processing system. The live data processing system comprises a live terminal 110, a live server 120, a first server 130, a second server 140 and a play terminal 150. Wherein the processing server includes a first server 130 and a second server 140, the processing server and the live broadcast server 120 may be integrated into one server or multiple servers, or may be separate servers. The terminal is connected with the server through a network, and the server is connected with the server through the network. The terminal may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a notebook computer, and the like, and further, live broadcast terminals 110 and the play terminal 150 are both installed with live broadcast related clients. The live broadcast segment in the playing terminal 150 that is playing live broadcast is obtained by pulling from the live broadcast segment of each second server 140, wherein the live broadcast segment of each second server 140 is obtained by filtering from a plurality of live broadcast segments that are pulled from the first server 130 and processed under a preset slicing condition, the first server 130 is connected with the live broadcast server 120, and the live broadcast stream data is pulled from the live broadcast server 120, and the live broadcast stream data of the live broadcast server 120 is pulled from the live broadcast stream data that is encoded and processed by the live broadcast terminal 110.

Specifically, the processing server receives live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data, and the processing server slices the live streaming data according to a preset slicing condition to obtain a plurality of corresponding live streaming fragments and caches the live streaming fragments. At the moment, the playing terminal sends a download request to the processing server, the processing server screens a plurality of currently cached live broadcast fragments according to the download request to obtain a target live broadcast fragment set, and finally, data in the target live broadcast fragment set is sent to the playing terminal by taking each live broadcast fragment as a unit. It should be noted that the server may be implemented as a stand-alone server or a server cluster composed of a plurality of servers.

As shown in fig. 2, in one embodiment, a live data processing method is provided. The embodiment is mainly illustrated by applying the method to the processing server in fig. 1. Referring to fig. 2, the live data processing method specifically includes the following steps:

step 202, receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data.

The live streaming data is data related to live broadcasting in a live broadcasting process, and includes but is not limited to a live data block composed of live video data and corresponding live audio data generated in the live broadcasting process, corresponding information data, a live video frame or a live audio frame itself and corresponding information data, and the like. It should be noted that the key frame time interval in the live streaming data can be set before the processing server receives the live streaming data, and the corresponding time stamp is carried in the live streaming data. Further, the live broadcast stream data not only includes live broadcast audio and video data, but also includes information data corresponding to each live broadcast audio and video data, where the information data is related information data required by live broadcast in a live broadcast process, and includes, but is not limited to, title information, text content information, numerical value transfer information, or live broadcast resource information, and the like required by the live broadcast process, and the information data may be represented by, but is not limited to, some private fields. The live streaming data may be, but is not limited to, a live audio/video data and corresponding information data, or a live video frame in the live video data and a corresponding live audio frame form a live streaming data block, and then the live streaming data block and the corresponding information data form live streaming data. In order to perform live broadcast on a play terminal, the processing server needs to receive live broadcast stream data related to the live broadcast, and specifically may receive the live broadcast stream data from another server or receive the live broadcast stream data from another terminal, where the another server may be the live broadcast server in fig. 1.

And 204, slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments.

The slicing condition is a condition for slicing live streaming data, and the slicing condition may be preset, and the specifically setting of the slicing condition includes but is not limited to slicing according to a group of pictures, GOP, of live video in the live streaming data, for example, one or more GOPs form a corresponding live slice, or slicing according to a preset slicing time interval, where the preset slicing time intervals corresponding to the live slices may be the same or different. In order to ensure the correct playing of the live video, the initial data frame of the video data corresponding to each live video segment is a key video frame. After the processing server obtains the preset slicing conditions, slicing the video key frames of the live broadcast stream data according to the obtained preset slicing conditions to obtain a plurality of corresponding live broadcast fragments. The live broadcast fragment refers to a live broadcast video fragment obtained by slicing live broadcast stream data, and the live broadcast fragment includes but is not limited to live broadcast audio and video related audio and video information, information data corresponding to live broadcast audio and video data, or live broadcast auxiliary broadcast information, and the like, wherein the live broadcast auxiliary broadcast information includes but is not limited to duration of the information data corresponding to the live broadcast audio and video data, and the like.

In one embodiment, when the preset slicing time interval is greater than the live video group of pictures GOP and the ratio of the preset slicing time interval to the live video group of pictures GOP in the live stream data is an integer, slicing is performed by taking the ratio of the preset slicing time interval to the live video group of pictures GOP as a unit, that is, one live slice has the same number of GOPs as the ratio of the preset slicing time interval to the live video group of pictures GOP.

In another embodiment, when the preset slicing time interval is greater than the live video group of pictures GOP and the ratio of the preset slicing time interval to the live video group of pictures GOP in the live stream data is a non-integer, the non-integer is rounded according to a preset method, for example, rounded down or rounded up to obtain a target integer, and then slicing is performed by using the target integer as a unit, that is, a target integer number of GOPs exist in one live slice.

In another embodiment, when the preset slicing time interval is smaller than the live video group of pictures GOP, the live video stream data is sliced by taking a single video group of pictures GOP as a unit to obtain a corresponding live slice, that is, only one GOP in one live slice is obtained.

And step 206, receiving a downloading request sent by the playing terminal, and screening the currently cached multiple live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set.

In one embodiment, the playing terminal watches live broadcast for the first time, and needs to send a download request to the processing server, where the download request is a request sent by the playing terminal to the processing server to download data related to the live broadcast. Generally, the broadcasting terminal may trigger a download request through a related live interface on a client in the broadcasting terminal, and send the download request to the processing server. Before receiving no download request sent by the playing terminal, the processing server caches a plurality of live broadcast fragments obtained by slicing live broadcast stream data according to preset slicing conditions in the processing server.

In another embodiment, when the play terminal watches the live broadcast, the play terminal is disconnected from the processing server due to a short abnormality of the network, so that the live broadcast of the play terminal cannot be continued. At this time, the playing terminal can also trigger a downloading request through a live interface where the live broadcast is located, and send the downloading request to the processing server. When the processing server is disconnected from the playing terminal, the previous live streaming data is cached in the processing server.

Specifically, once the processing server receives a download request sent by the play terminal, the processing server needs to screen a plurality of currently cached live broadcast segments according to the download request to obtain target live broadcast segments, and the target live broadcast segments form a target live broadcast segment set, where a specific rule of the screening may be self-defined. The customization may be, but is not limited to, returning a preset time interval, or returning a preset number of screened live broadcast fragments, and the like. Specifically, the processing server takes the time point when the download request sent by the playing terminal is received as a screening starting time point, and then screens the live broadcast fragments before the time of the screening starting time point according to the screening rule to obtain target live broadcast fragments according with the screening rule, and finally forms the target live broadcast fragments obtained through screening into a target live broadcast fragment set.

If the filtering rule is that 3 pieces of data of the preset filtering live broadcast fragments are set for backspacing, the processing server takes the time point of the received download request sent by the playing terminal as the filtering starting time point, and filters 3 live broadcast fragments forwards from a plurality of currently cached live broadcast fragments according to the filtering starting time point, namely 3 live broadcast fragments closest to the filtering starting time point, then takes the 3 live broadcast fragments obtained by filtering as target live broadcast fragments, and finally forms the 3 target live broadcast fragments into a target live broadcast fragment set.

And step 208, sending the data in the target live broadcast segment set to the playing terminal by taking each live broadcast segment as a unit, so that the playing terminal starts to play the received live broadcast segment when the received live broadcast segment meets a preset playing condition.

Specifically, in order to make the playing of the playing terminal more accurate and improve the consistency of each playing terminal, the player of the playing terminal includes, but is not limited to, a function of controlling the amount of the live data cached in the player, or a function of controlling the playing duration of the live data in the player, and the like. Therefore, the processing server needs to send the data in the target live broadcast segment set to the playing terminal by taking each live broadcast segment as a unit, and when the live broadcast segment received by the playing terminal meets the preset playing condition, the received live broadcast segment can be played. The preset playing condition is a playing condition which can be preset in the player and is used for controlling the received live broadcast fragments, and includes but is not limited to detecting that the number of video frames in the received live broadcast fragments reaches a preset video frame number threshold value, or detecting that the number of the received live broadcast fragments reaches a preset live broadcast fragment number and the like.

If the preset playing condition is that the number of video frames in the live broadcast fragments received by the player reaches 40 frames of video frames, the processing server sends the data in the target live broadcast fragment set to the playing terminal in units of each live broadcast fragment, and the playing terminal does not meet 40 frames of video frames when receiving the first live broadcast fragment, so that the second live broadcast fragment is continuously received, the preset playing condition is met when receiving approximately half of the second live broadcast fragment, and the playing terminal can start playing the received live broadcast fragment.

According to the live broadcast data processing method, a processing server receives live broadcast stream data comprising live broadcast audio and video data and corresponding information data, the live broadcast stream data is sliced according to preset slicing conditions to obtain a plurality of live broadcast fragments and is cached, a downloading request sent by a playing terminal is received, the plurality of currently cached live broadcast fragments are screened according to the downloading request to obtain a target live broadcast fragment set, and finally data in the target live broadcast fragment set are sent to the playing terminal by taking each live broadcast fragment as a unit, so that the playing terminal starts to play the received live broadcast fragments when the received live broadcast fragments meet the preset playing conditions. The time delay error brought by the processing server can be controlled through the preset slicing condition and the preset condition, and the playing terminal can play the audio only after meeting the preset playing condition, so that the playing time error between the live broadcasting audience terminals is reduced, and the consistency between the playing terminals is improved.

As shown in fig. 3, in an embodiment, on the basis of the live data processing method shown in fig. 2, the method further includes:

step 302, receiving live audio and video data and corresponding information data, wherein the live audio and video data comprises a plurality of live data blocks.

Specifically, the processing server may receive live streaming data sent by the live streaming server, or may receive live streaming data sent by other terminals, and then when the processing server receives the live streaming data, the live streaming server performs related processing on the live streaming data. Specifically, the live broadcast server receives live broadcast audio and video data and corresponding information data sent by a live broadcast terminal, the live broadcast server combines each live broadcast video frame and a corresponding live broadcast audio frame in the live broadcast audio and video data into each live broadcast data block, or the live broadcast server combines all live broadcast video frames or all live broadcast audio frames in the live broadcast audio and video data into each live broadcast data block, and the like, and the live broadcast data block can be but is not limited to a single live broadcast video frame and a corresponding live broadcast audio frame, or a multi-frame live broadcast video frame and a corresponding live broadcast audio frame, and the like.

And 304, acquiring the synchronization information corresponding to each live video frame from the information data, and packaging the synchronization information to a live data block where the corresponding live video frame is located to form live streaming data.

Specifically, in order to enable the playing terminal to synchronously play or synchronously display live audio and video data and corresponding information data when the playing terminal performs live playing, after the live playing server receives the live audio and video data and the corresponding information data, the live playing server obtains synchronization information corresponding to each frame of live video frames in the live audio and video data, namely obtains synchronization information corresponding to each frame of live video frames in the live audio and video data from the information data. Further, the live broadcast server packages the synchronization information into live broadcast stream data formed by the live broadcast data block where the corresponding live broadcast video frame is located. Since the information data can be represented by a private field, the encapsulation can be, but is not limited to, encapsulating the synchronization information in a live data block header in a live data block where a live video frame corresponding to the synchronization information is located, or at the end of the live data block, or in a specific position of the live data block. And the live broadcast server forms live broadcast stream data by packaging the updated live broadcast data blocks and then sends the live broadcast stream data to the processing server.

According to the live broadcast data processing method, the synchronization information corresponding to each frame of live broadcast video frame in the live broadcast audio and video frame data is packaged to the live broadcast data block where the corresponding live broadcast video frame is located to form live broadcast stream data, so that the live broadcast audio and video data and the corresponding information data can be completely synchronized, when the broadcast terminal receives the live broadcast fragment, the live broadcast audio and video data and the information data are analyzed, and the live broadcast audio and video data and the corresponding information data are synchronously broadcast. Therefore, synchronous playing between the live audio and video data and the corresponding information data is achieved on the basis of reducing the time delay error between the live audience terminals.

As shown in fig. 4, in an embodiment, slicing live streaming data according to a preset slicing condition to obtain a plurality of corresponding live slices includes:

step 402, acquiring a preset slicing time interval.

Step 404, obtaining a key frame duration interval corresponding to each video image group in the live streaming data.

Instep 406, the key frame slicing interval is determined according to the preset slicing time interval and the key frame duration interval.

And step 408, slicing the live broadcast stream data according to the key frame slicing interval to obtain a plurality of live broadcast slices.

The preset slicing time interval is a slicing time interval for slicing live streaming data preset by the processing server, the key frame time interval refers to an interval of key frame time lengths corresponding to each video image group in the live streaming data, and a video image group is an interval between two video key frames and can be called a GOP. Because the processing server sets the time interval between the live video key frames before receiving the live streaming data and the live broadcast terminal encodes the collected live audio and video data, the time interval between the live video key frames can be set to be smaller, and then the live video data with the updated key frame time interval is encoded.

In one embodiment, the processing server obtains a preset slicing time interval for slicing the live streaming data, and when the preset slicing time interval is greater than a live video group of pictures (GOP) and the ratio of the preset slicing time interval to the GOP in the live streaming data is an integer, the processing server slices the live streaming data in units of the ratio of the preset slicing time interval to the GOP, that is, one live streaming slice has GOPs with the same number as the ratio of the preset slicing time interval to the GOP.

If the preset segmentation time interval is 2s, and the time interval of the video key frames in the live streaming data is 1s, that is, the video group of pictures GOP is 1s, the live streaming data is segmented according to the preset segmentation condition to obtain each live segment, and each live segment includes an interval of two live video key frames, that is, two video group of pictures GOP are in one live segment.

In another embodiment, when the preset slicing time interval is greater than the live video group of pictures GOP and the ratio of the preset slicing time interval to the live video group of pictures GOP in the live stream data is a non-integer, the non-integer is rounded according to a preset method, for example, rounded down or rounded up to obtain a target integer, and then slicing is performed by using the target integer as a unit, that is, a target integer number of GOPs exist in one live slice.

If the preset slicing condition is that the preset slicing time interval is 7s, and the video group of pictures (GOP) in the live streaming data is 2s, because the ratio of the preset slicing time interval to the GOP in the live streaming data is a non-integer, rounding the non-integer according to a preset method, obtaining a target integer if rounding downwards or rounding upwards, and slicing by taking the target integer as a unit, wherein each live streaming segment comprises three video group of pictures (GOPs) or each live streaming segment comprises four video group of pictures (GOPs).

In another embodiment, when the preset slicing time interval is smaller than the live video group of pictures GOP, the live video stream data is sliced by taking a single video group of pictures GOP as a unit to obtain a corresponding live slice, that is, only one GOP in one live slice is obtained.

If the preset slicing condition is that the preset slicing time interval is 1.5s and the video group of pictures (GOP) in the live streaming data is 2s, slicing the live streaming data by taking the GOP as a unit to obtain each live streaming fragment, namely each live streaming fragment comprises one video group of pictures (GOP).

As shown in fig. 5, in an embodiment, the screening a plurality of currently cached live broadcast segments according to a download request to obtain a target live broadcast segment set includes:

step 502, acquiring a preset screening number, screening live broadcast fragments with the preset screening number from a plurality of currently cached live broadcast fragments according to a downloading request to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set.

Specifically, after the processing server receives the download request, because the processing server presets a screening strategy for screening target live broadcast fragments from a plurality of currently cached live broadcast fragments, the processing server first needs to obtain a preset screening number in the screening strategy, then takes a time point at which the download request sent by the play terminal is received as a screening start time point, screens the live broadcast fragments with the preset screening number from the live broadcast fragments cached in the processing server before the screening start time point, further obtains the target live broadcast fragments, and then forms the target live broadcast fragments obtained by screening into a target live broadcast fragment set.

If the preset screening number is 3, which is obtained from the screening strategy preset by the processing server, the time point at which the download request sent by the playing terminal is received is taken as the screening starting time point, 3 live broadcast fragments are screened forward from the live broadcast fragments cached in the processing server from the screening starting time point to obtain corresponding target live broadcast fragments, namely 3 live broadcast fragments closest to the screening starting time point are taken as target live broadcast fragments, and finally the 3 target live broadcast fragments form a target live broadcast fragment set.

And 504, acquiring a preset back-off time interval, and backing off the preset back-off time interval according to the downloading request to obtain a starting point of the starting live broadcast fragment.

Step 506, taking the start live broadcast segment corresponding to the start point of the start live broadcast segment and the live broadcast segment data after the start live broadcast segment as target live broadcast segments, and forming the target live broadcast segments into a target live broadcast segment set.

In this embodiment, specifically, the processing server presets to screen a plurality of currently cached live broadcast segments, so as to obtain a target live broadcast segment. The method comprises the steps of caching a plurality of live broadcast fragments, caching the live broadcast fragments, and returning the live broadcast fragments to a current cache according to a preset time interval, wherein the time interval for returning the live broadcast fragments can be preset, and because the structure in the live broadcast stream data carries a timestamp, the time point of receiving a download request sent by a playing terminal can be obtained first, and then the time point is used as a reference point, and the preset time interval is returned to the historical time direction to obtain the starting point of the initial live broadcast fragments. Further, the start live broadcast segment corresponding to the start point of the start live broadcast segment and the live broadcast segment data after the start live broadcast segment are used as target live broadcast segments, and finally the target live broadcast segments form a target live broadcast segment set.

If the preset back-off time interval is obtained from the preset screening strategy of the processing server and the screening starting time point of the processing server receiving the download request sent by the playing terminal is 5s, the processing server backs off the preset back-off time interval 1s from the screening starting time point to obtain a starting live broadcast segment starting point 4s, then the starting live broadcast segment corresponding to the starting live broadcast segment starting point 4s and live broadcast segment data after the starting live broadcast segment are used as target live broadcast segments, and finally the target live broadcast segments form a target live broadcast segment set.

It should be noted that, the relationship betweenstep 502 and step 504 and step 506 is, that is, only step 502 may be executed, or only step 504 and step 506 may be executed, or step 502,step 504 and step 506 may be executed simultaneously, or step 502 is executed beforestep 504 and step 506 are executed.

As shown in fig. 6, in an embodiment, data in a target live broadcast segment set is sent to a playback terminal in units of live broadcast segments, so that the playback terminal starts playing a received live broadcast segment when the received live broadcast segment meets a preset playing condition, which is mainly exemplified by applying the method to the playback terminal 110 in fig. 1. The method comprises the following steps:

step 602, the playing terminal analyzes the received live broadcast segment and determines the number of live broadcast video frames corresponding to the received live broadcast segment.

And step 604, when the number of the live broadcast video frames meets a preset video frame number threshold, starting to play the received live broadcast fragments.

In this embodiment, the processing server sends the data in the target live broadcast segment set to the playing terminal in units of each live broadcast segment, as described above, the player on the playing terminal has a function of controlling the amount of the live broadcast data cached in the player, or a function of controlling the playing duration of the live broadcast data in the player, and so on. Therefore, when the playing terminal needs to detect the received live broadcast fragment sent by the processing server, specifically, the playing terminal needs to analyze the live broadcast fragment sent by the processing server to obtain the number of live broadcast video frames corresponding to the received live broadcast fragment, and when the number of live broadcast video frames meets a preset video frame number threshold, the playing terminal starts to play the received live broadcast fragment.

As shown in fig. 7, fig. 7 is a schematic diagram illustrating that a playing terminal plays a received live broadcast segment in an embodiment, after receiving a download request sent by the playing terminal, a processing server obtains corresponding target live broadcast segments t7 and t6 by filtering forward from a time point at which the download request is received from among the cached live broadcast segments according to preset conditions, the target live broadcast segments t7 and t6 form a target live broadcast segment set, and then data in the target live broadcast segment set is sent to the playing terminal in units of each live broadcast segment, that is, the live broadcast segment t7 is sent to the playing terminal first, and after receiving t7, the playing terminal analyzes the live broadcast segment t7 to determine the number of live broadcast video frames corresponding to the live broadcast segment t 7. When the number of live video frames in the live video segment t7 does not meet the preset video frame number threshold, continuing to receive the live video segment t6, analyzing the live video segment t6, determining the number of live video frames corresponding to the live video segment t6, and when the number of live video frames in the live video segment t7 and part of live video frames in the live video segment t6 reach the preset video frame number threshold, taking the time point of the part of live video frames in the live video segment t6 as a live broadcast playing point, and starting to play the received live video segment.

As shown in fig. 7, when the network of the playback terminal is abnormal, because the time length set by the video image group in the live streaming data is short, and the time length of the live streaming fragment obtained by segmenting the live streaming data according to the preset condition is also short, when the network of the playback terminal is recovered to be normal, the playback terminal can well catch up with the playback content, and it can avoid that the delay time of each playback terminal increases due to a long-time playback delay of a user of the playback terminal under a weak network condition.

Furthermore, the playing terminal analyzes the live broadcast video frame number corresponding to the live broadcast segment in the live broadcast segment, also analyzes the synchronization information corresponding to the live broadcast video frame in the received live broadcast segment, and synchronously plays or synchronously displays the analyzed synchronization information and the corresponding live broadcast video frame. For example, in the application scenario of live broadcast answering, the play terminal starts playing the live broadcast segment meeting the preset playing condition, that is, when the live broadcast audio/video frame in the live broadcast segment says a similar utterance after answering the question for the host, the play terminal synchronizes the synchronization information corresponding to the live broadcast audio/video frame, such as the question 1, at the play terminal for the audience of the play terminal to perform live broadcast answering.

In an application scene of live broadcast answer, a live broadcast terminal encodes live broadcast audio and video data related to live broadcast, the encoded live broadcast audio and video data and corresponding live broadcast title information are sent to a live broadcast server together, the live broadcast server receives the live broadcast audio and video data and the corresponding live broadcast title information and sends the live broadcast audio and video data and the corresponding live broadcast title information as live broadcast stream data to a first server together, and the live broadcast server can correct the live broadcast audio and video data and the corresponding live broadcast title information slightly and the like. The method comprises the steps that after receiving live audio and video data and corresponding live topic information data, a first server slices live streaming data according to preset slicing conditions to obtain a plurality of corresponding live fragments, and sends the live fragments to a second server, and the second server receives the live fragments and caches the live fragments to the second server. When a playing terminal where the audience is located sends a downloading request to a second server, the second server screens a plurality of currently cached live broadcast fragments according to a pre-downloading request to obtain target live broadcast fragments, and the target live broadcast fragments form a target live broadcast fragment set. And finally, sending the data in the target live broadcast segment set to the playing terminal of each audience by taking each live broadcast segment as a unit.

Further, the playing terminal needs to detect whether the received live broadcast segment meets a preset playing condition, and if the preset playing condition is met, the playing terminal starts to play the received live broadcast segment, that is, play the live broadcast audio and video data in the live broadcast segment and the corresponding live broadcast title information, as shown in fig. 8a, fig. 8a is an application scene diagram of the live broadcast data processing method in one embodiment, and when a host in the live broadcast terminal says "please listen to the title", a live broadcast interface of the playing terminal displays the live broadcast title information corresponding to the live broadcast audio and video data for audiences to perform live broadcast answer. By applying the live broadcast data processing method to a live broadcast answer application scene, the play delay among the playing terminals can be reduced through the optimization of the first server and the second server, and the delay time of each playing terminal can be kept less than 2 s.

In an application scene of live broadcast resource transfer, a live broadcast terminal encodes live broadcast audio and video data related to live broadcast, and transmits the encoded live broadcast audio and video data and corresponding live broadcast resource transfer information, such as live broadcast red packet information, to a live broadcast server together. The method comprises the steps that after receiving live audio and video data and corresponding live resource transfer information, a first server slices live streaming data according to preset slicing conditions to obtain a plurality of corresponding live streaming fragments, and sends the live streaming fragments to a second server, and the second server receives the live streaming fragments and caches the live streaming fragments to the second server. When a playing terminal where the audience is located sends a downloading request to a second server, the second server screens a plurality of currently cached live broadcast fragments according to the downloading request to obtain target live broadcast fragments, and the target live broadcast fragments form a target live broadcast fragment set. And finally, sending the data in the target live broadcast segment set to the playing terminal of each audience by taking each live broadcast segment as a unit.

Further, the playing terminal needs to detect whether the received live broadcast fragment meets a preset playing condition, and if the preset playing condition is met, the playing terminal starts to play the received live broadcast fragment, namely, the live broadcast audio and video data in the live broadcast fragment and the corresponding live broadcast resource transfer information are played. As shown in fig. 8b, fig. 8b is an application scenario diagram of a live data processing method in an embodiment, when a host in a live terminal says "prepare start," a play terminal displays live resource transfer information corresponding to live audio and video data on a live interface for viewers to transfer, such as live red packet information. By applying the live data processing method to the application scene of live resource transfer, the playing delay among the playing terminals can be reduced through the optimization of the first server and the second server, and the delay time of each playing terminal can be kept less than 2 s.

Fig. 8 is a timing diagram of a live data processing method in an embodiment, including a live terminal, a live server, a processing server, and a play terminal, where the processing server includes a first server and a second server. Specifically, the live broadcast terminal sends live broadcast audio and video data related to live broadcast to the live broadcast server, and meanwhile, operators related to a live broadcast platform send information data corresponding to the live broadcast audio and video data to the live broadcast server together. After the live broadcast audio and video data and the corresponding information data are received by the live broadcast server, the live broadcast audio and video data comprise a plurality of live broadcast data blocks, so that synchronous information corresponding to each frame of live broadcast video frame is obtained first, then the synchronous information is packaged to the live broadcast data block where the corresponding live broadcast video frame is located to form live broadcast stream data, and finally the live broadcast server sends the live broadcast stream data to the first server. After receiving the live streaming data sent by the live streaming server, the first server slices the live streaming data according to preset slicing conditions to obtain a plurality of corresponding live streaming fragments, and then distributes the plurality of live streaming fragments to the second server.

The second server needs to cache the plurality of live broadcast fragments sent by the first server before receiving the download request sent by the playing terminal, and once the second server receives the download request sent by the playing terminal, the second server needs to screen the plurality of currently cached live broadcast fragments according to the download request to obtain target live broadcast fragments, and the target live broadcast fragments form a target live broadcast fragment set. And the second server sends the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit, the playing terminal analyzes the received live broadcast fragments to determine the number of live broadcast video frames corresponding to the received live broadcast fragments, when the number of the live broadcast video frames does not meet a preset video frame number threshold value, the playing terminal continues to receive the next live broadcast fragment and then analyzes the next live broadcast fragment, and the playing terminal starts to play the received live broadcast fragments until the number of the live broadcast video frames in the live broadcast fragments in the playing terminal meets the preset video frame number threshold value.

In a specific embodiment, a live data processing method is provided, which specifically includes the following steps:

and step 902, other servers, such as a live broadcast server, receive live broadcast audio and video data sent by a live broadcast terminal and information data corresponding to the live broadcast audio and video data sent by an operator corresponding to a live broadcast platform, wherein the live broadcast audio and video data comprises a plurality of live broadcast data blocks.

And 904, after receiving the live audio and video data and the corresponding information data, other servers acquire the synchronization information corresponding to each frame of live video frame, package the synchronization information to a live data block where the corresponding live video frame is located to form live streaming data, and send the live streaming data to a processing server.

Step 906, the processing server receives live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data.

Step 908, the processing server first obtains a preset slicing time interval, then obtains a key frame time interval corresponding to each video image group in the live streaming data, then determines a key frame slicing interval according to the preset slicing time interval and the key frame time interval, and finally slices the live streaming data according to the key frame slicing interval to obtain a plurality of live streaming fragments.

Step 910, the processing server receives a download request sent by the play terminal, and filters the currently cached multiple live broadcast segments according to the download request to obtain a corresponding target live broadcast segment set.

Step 910a, the processing server obtains a preset screening number according to the download request, screens live broadcast fragments with the preset screening number from the currently cached multiple live broadcast fragments according to the download request to obtain target live broadcast fragments, and forms the target live broadcast fragments into a target live broadcast fragment set.

Step 910b, the processing server obtains a preset back-off time interval according to the download request, backs off the preset back-off time interval according to the download request to obtain a start point of the start live broadcast fragment, takes the start live broadcast fragment corresponding to the start point of the start live broadcast fragment and the live broadcast fragment data after the start live broadcast fragment as target live broadcast fragments, and combines the target live broadcast fragments into a target live broadcast fragment set.

Step 912, the processing server sends the data in the target live broadcast segment set to the playing terminal in units of each live broadcast segment, when the playing terminal receives the first live broadcast segment, the live broadcast segment is analyzed to obtain the number of live broadcast video frames corresponding to the live broadcast segment, if the number of live broadcast video frames does not meet the preset video frame number threshold, the playing terminal continues to receive the next live broadcast segment, and then analyzes the next live broadcast segment to obtain the corresponding number of live broadcast video frames, until the number of live broadcast video frames analyzed in the playing terminal reaches the preset video frame number threshold, the received live broadcast segment starts to be played.

It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the above-described flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least a portion of the sub-steps or stages of other steps.

As shown in fig. 9, in one embodiment, there is provided a live data processing system comprising:

theprocessing server 1002 is configured to receive live streaming data, where the live streaming data includes live audio and video data and corresponding information data, and slice the live streaming data according to a preset slicing condition to obtain a plurality of corresponding live slices.

The processing server comprises a first server and a second server, wherein the first server is used for receiving live streaming data of live audio and video data and corresponding information data, the live streaming data is data related to live broadcasting in a live broadcasting process, and the live streaming data comprises but is not limited to a live data block consisting of live video data and corresponding live audio data generated in the live broadcasting process, corresponding information data, a live video frame or a live audio frame and corresponding information data and the like. And after receiving the live streaming data, the first server needs to slice the live streaming data according to a preset slicing condition to obtain a plurality of corresponding live streaming fragments. The slicing condition is a condition for slicing live streaming data, and the slicing condition may be set in advance, including but not limited to slicing according to a group of pictures, GOPs, of live video in the live streaming data, for example, one or more GOPs form a corresponding live slice, or slicing according to a preset slicing time interval, where the preset slicing time interval corresponding to each live slice may be the same or different. In order to ensure the correct playing of the live video, the initial data frame of the video data corresponding to each live video segment is a key video frame.

After the first server obtains the corresponding live broadcast fragments, the corresponding live broadcast fragments are sent to a second server, and the second server needs to cache the received live broadcast fragments into the second server before receiving the download request sent by the playing terminal.

Theplay terminal 1004 is configured to send a download request to the processing server.

Theprocessing server 1002 is further configured to receive a download request sent by the play terminal, screen a plurality of currently cached live broadcast segments according to the download request to obtain a target live broadcast segment set, and send data in the target live broadcast segment set to the play terminal in units of each live broadcast segment.

In one embodiment, the first time the broadcast terminal watches the live broadcast, a download request needs to be sent to the second server, and the download request is a request sent by the broadcast terminal to the second server to download data related to the live broadcast. In general, the cast terminal may trigger a download request through an associated live interface and send the download request to the second server. When the second server does not receive the downloading request sent by the playing terminal, the second server can cache a plurality of live broadcast fragments obtained by slicing the live broadcast stream data according to the preset slicing conditions in the second server.

In another embodiment, when the broadcast terminal watches the live broadcast, the broadcast terminal is disconnected from the second server due to a short abnormality of the network, so that the live broadcast of the broadcast terminal cannot be continued. At this time, the playing terminal can also trigger a downloading request through a live interface where the live broadcast is located, and send the downloading request to the second server. When the second server is disconnected from the playing terminal, the previous live streaming data can be cached in the second server.

Once a second server in the processing server receives a downloading request sent by the playing terminal, a plurality of live broadcast fragments cached in the second server are screened according to the downloading request to obtain target live broadcast fragments, the target live broadcast fragments form a target live broadcast fragment set, and finally data in the target live broadcast fragment set are sent to the playing terminal by taking each live broadcast fragment as a unit.

The playing terminal 1004 is further configured to detect whether the received live broadcast segment sent by the processing server meets a preset playing condition, and start playing the received live broadcast segment if the preset playing condition is met.

Specifically, when the playing terminal needs to detect whether the received live broadcast segment meets a preset playing condition, if the preset playing condition is met, the playing terminal can start playing the received live broadcast segment. The preset playing condition is a playing condition which can be preset in the player and is used for controlling the received live broadcast fragments, and includes but is not limited to detecting that the number of video frames in the received live broadcast fragments reaches a preset video frame number threshold value, or detecting that the number of the received live broadcast fragments reaches a preset live broadcast fragment number and the like.

As shown in fig. 10, in one embodiment, the system further comprises: alive server 1006, wherein:

thelive broadcast server 1006 is configured to receive live broadcast audio/video data and corresponding information data, where the live broadcast audio/video data includes a plurality of live broadcast data blocks.

Thelive broadcast server 1006 is further configured to obtain synchronization information corresponding to each live broadcast video frame from the information data, and encapsulate the synchronization information into a live broadcast data block in which the corresponding live broadcast video frame is located to form live broadcast stream data.

Specifically, the live broadcast data processing system further comprises a live broadcast server, the live broadcast server establishes network connection with a first server in the processing server, the live broadcast server receives live broadcast audio and video data and corresponding information data sent by a live broadcast terminal, the live broadcast server needs to combine each live broadcast video frame and a corresponding live broadcast audio frame in the live broadcast audio and video data into each live broadcast data block, or the live broadcast server combines all live broadcast video frames or all live broadcast audio frames in the live broadcast audio and video data into each live broadcast data block, and the like, and the live broadcast data block can be but is not limited to one frame consisting of each live broadcast video frame and a corresponding live broadcast audio frame, or multiple frames consisting of live broadcast video frame frames and corresponding live broadcast audio frames, and the like.

Further, after the live broadcast server receives the live broadcast audio/video data and the corresponding information data, the live broadcast server needs to acquire synchronization information corresponding to each frame of live broadcast video frame in the live broadcast audio/video data, that is, synchronization information corresponding to each frame of live broadcast video frame in the live broadcast audio/video data is acquired from the information data. And the live broadcast server packages the synchronous information into live broadcast stream data formed by the live broadcast data block where the corresponding live broadcast video frame is located. Since the information data can be represented by a private field, the encapsulation can be, but is not limited to, encapsulating the synchronization information in a live data block header in a live data block where a live video frame corresponding to the synchronization information is located, or at the end of the live data block, or in a specific position of the live data block. And the live broadcast server forms live broadcast stream data by each live broadcast data block after being packaged and updated and sends the live broadcast stream data to a packaging server in the processing server.

In one embodiment, theprocessing server 1002 is further configured to obtain a preset slicing time interval; theprocessing server 1002 is further configured to obtain a key frame duration interval corresponding to each video image group in the live streaming data; theprocessing server 1002 is further configured to determine a key frame slicing interval according to a preset slicing time interval and a key frame duration interval; theprocessing server 1002 is further configured to slice the live streaming data according to the key frame slicing interval to obtain a plurality of live streaming sliced data.

In an embodiment, theprocessing server 1002 is further configured to obtain a preset screening number, and screen live broadcast segment data of the preset screening number in a plurality of currently cached live broadcast segments according to a download request to obtain target live broadcast segment data; and/or theprocessing server 1002 is further configured to obtain a preset backoff time interval, back the preset backoff time interval according to the download request to obtain a start point of the start live broadcast segment, and compose start live broadcast segment data corresponding to the start point of the start live broadcast segment and live broadcast segment data subsequent to the start live broadcast segment data into target live broadcast segment data.

In one embodiment, the playing terminal 1004 is further configured to parse the received live broadcast segment, and determine a number of live broadcast video frames corresponding to the received live broadcast segment; the playing terminal 1004 is further configured to start playing the received live broadcast segment when the number of live broadcast video frames meets a preset video frame number threshold.

As shown in fig. 11, in one embodiment, there is provided a live data processing apparatus 1100, the apparatus comprising:

a live streamingdata receiving module 1102, configured to receive live streaming data, where the live streaming data includes live audio and video data and corresponding information data;

the live streamingdata slicing module 1104 is configured to slice live streaming data according to a preset slicing condition to obtain a plurality of corresponding live streaming fragments;

a downloadrequest receiving module 1106, configured to receive a download request sent by a play terminal;

a target live broadcastfragment screening module 1108, configured to screen multiple currently cached live broadcast fragments according to the download request to obtain a target live broadcast fragment set;

the live broadcastsegment sending module 1110 is configured to send data in the target live broadcast segment set to the play terminal in units of each live broadcast segment, so that the play terminal starts to play the received live broadcast segment when the received live broadcast segment meets a preset play condition.

As shown in fig. 12, in one embodiment, the live data processing apparatus 1100 further includes:

a live audio and videodata receiving module 1202, configured to receive live audio and video data and corresponding information data, where the live audio and video data includes a plurality of live data blocks;

and a synchronizationinformation obtaining module 1204, configured to obtain synchronization information corresponding to each frame of live video frame, and encapsulate the synchronization information into a live data block where the corresponding live video frame is located to form live streaming data.

As shown in fig. 13, in one embodiment, the live streamdata slicing module 1104 includes:

a slicing timeinterval acquisition unit 1104a for acquiring a preset slicing time interval;

a key frameduration obtaining unit 1104b, configured to obtain a key frame duration interval corresponding to each video image group in the live streaming data;

a key frame slicinginterval generating unit 1104c for determining a key frame slicing interval according to a preset slicing time interval and a key frame duration interval;

and the live broadcast fragmentdata generating unit 1104d is configured to slice live broadcast stream data according to the key frame slicing interval to obtain a plurality of live broadcast fragment data.

As shown in fig. 14, in one embodiment, target livesegment filtering module 1108 includes:

a preset screeningnumber obtaining unit 1108a, configured to obtain a preset screening number;

a target live broadcast segmentset generating unit 1108b, configured to screen live broadcast segments with a preset screening number from a plurality of currently cached live broadcast segments according to a download request to obtain target live broadcast segments, and form the target live broadcast segments into a target live broadcast segment set; and/or

A preset back-off timeinterval obtaining unit 1108c, configured to obtain a preset back-off time interval, and obtain a starting point of an initial live broadcast segment according to a download request back-off the preset back-off time interval;

and a target live broadcast segment set acquiring unit 1108d, configured to combine the start live broadcast segment corresponding to the start point of the start live broadcast segment and the live broadcast segment data after the start live broadcast segment into a target live broadcast segment, and combine the target live broadcast segment into a target live broadcast segment set.

In one embodiment, the playing terminal analyzes the received live broadcast fragments and determines the number of live broadcast video frames corresponding to the received live broadcast fragments; and when the number of the live video frames meets a preset video frame number threshold, starting to play the received live video fragments.

FIG. 15 is a diagram showing an internal structure of a computer device in one embodiment. The computer device may specifically be the processing server in fig. 1. As shown in fig. 15, the computer apparatus includes a processor, a memory, a network interface, and an input device connected via a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by the processor, causes the processor to implement a live data processing method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform a live data processing method. The input device of the computer equipment can be a touch layer covered on a display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 15 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the live data processing apparatus provided in the present application may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in fig. 15. The memory of the computer device may store various program modules constituting the live data processing apparatus, such as a live streaming data receiving module, a live streaming data slicing module, a download request receiving module, a target live streaming segment screening module, and a live streaming segment sending module shown in fig. 11. The computer program constituted by the respective program modules causes the processor to execute the steps in the live data processing method of the embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 15 may perform receiving live streaming data by a live streaming data receiving module in the live data processing apparatus shown in fig. 11, where the live streaming data includes live audio and video data and corresponding information data. The computer equipment can slice the live streaming data according to preset slicing conditions through the live streaming data slicing module to obtain a plurality of corresponding live streaming fragments.

In one embodiment, a computer device is proposed, comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of: receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data; slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments; receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set; and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

In one embodiment, the computer program further causes the processor to perform the steps of: receiving live audio and video data and corresponding information data, wherein the live audio and video data comprises a plurality of live data blocks; and acquiring the synchronization information corresponding to each frame of live video frame from the information data, and packaging the synchronization information to the live data block where the corresponding live video frame is located to form live stream data.

In one embodiment, slicing live streaming data according to a preset slicing condition to obtain a plurality of corresponding live slices includes: acquiring a preset slicing time interval; acquiring a key frame duration interval corresponding to each video image group in live streaming data; determining a key frame slicing interval according to a preset slicing time interval and a key frame duration interval; and slicing the live broadcast stream data according to the key frame slicing interval to obtain a plurality of live broadcast slices.

In one embodiment, the screening of the currently cached multiple live broadcast segments according to the download request to obtain the target live broadcast segment set includes: acquiring a preset screening number, screening live broadcast fragments with the preset screening number from a plurality of currently cached live broadcast fragments according to a downloading request to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set; and/or obtaining a preset backspacing time interval, and backspacing the preset backspacing time interval according to the downloading request to obtain a starting point of the starting live broadcast fragment; and taking the start live broadcast fragment corresponding to the start point of the start live broadcast fragment and the live broadcast fragment data after the start live broadcast fragment as target live broadcast fragments, and forming a target live broadcast fragment set by the target live broadcast fragments.

In one embodiment, the method for transmitting data in a target live broadcast segment set to a play terminal in units of live broadcast segments so that the play terminal starts to play a received live broadcast segment when the received live broadcast segment meets a preset play condition includes: the playing terminal analyzes the received live broadcast fragments and determines the number of live broadcast video frames corresponding to the received live broadcast fragments; and when the number of the live video frames meets a preset video frame number threshold, starting to play the received live video fragments.

In one embodiment, a computer-readable storage medium is proposed, in which a computer program is stored which, when executed by a processor, causes the processor to carry out the steps of: receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data; slicing the live broadcast stream data according to preset slicing conditions to obtain a plurality of corresponding live broadcast fragments; receiving a downloading request sent by a playing terminal, and screening a plurality of currently cached live broadcast fragments according to the downloading request to obtain a target live broadcast fragment set; and sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit so that the playing terminal starts to play the received live broadcast fragment when the received live broadcast fragment meets the preset playing condition.

In one embodiment, the computer program further causes the processor to perform the steps of: receiving live audio and video data and corresponding information data, wherein the live audio and video data comprises a plurality of live data blocks; and acquiring the synchronization information corresponding to each frame of live video frame from the information data, and packaging the synchronization information to the live data block where the corresponding live video frame is located to form live stream data.

In one embodiment, slicing live streaming data according to a preset slicing condition to obtain a plurality of corresponding live slices includes: acquiring a preset slicing time interval; acquiring a key frame duration interval corresponding to each video image group in live streaming data; determining a key frame slicing interval according to a preset slicing time interval and a key frame duration interval; and slicing the live broadcast stream data according to the key frame slicing interval to obtain a plurality of live broadcast slices.

In one embodiment, the screening of the currently cached multiple live broadcast segments according to the download request to obtain the target live broadcast segment set includes: acquiring a preset screening number, screening live broadcast fragments with the preset screening number from a plurality of currently cached live broadcast fragments according to a downloading request to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set; and/or obtaining a preset backspacing time interval, and backspacing the preset backspacing time interval according to the downloading request to obtain a starting point of the starting live broadcast fragment; and taking the start live broadcast fragment corresponding to the start point of the start live broadcast fragment and the live broadcast fragment data after the start live broadcast fragment as target live broadcast fragments, and forming a target live broadcast fragment set by the target live broadcast fragments.

In one embodiment, the method for transmitting data in a target live broadcast segment set to a play terminal in units of live broadcast segments so that the play terminal starts to play a received live broadcast segment when the received live broadcast segment meets a preset play condition includes: the playing terminal analyzes the received live broadcast fragments and determines the number of live broadcast video frames corresponding to the received live broadcast fragments; and when the number of the live video frames meets a preset video frame number threshold, starting to play the received live video fragments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (15)

1. A live data processing method includes:

receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data, and the information data comprises at least one of title information, text content information, numerical value transfer information and live resource information required in a live broadcasting process;

acquiring a preset slicing time interval;

acquiring a key frame duration interval corresponding to each video image group in live streaming data;

determining a key frame slicing interval according to the preset slicing time interval and the key frame duration interval, wherein the preset slicing time interval is greater than the key frame duration interval, and the key frame slicing interval is greater than the key frame duration interval;

slicing the live broadcast stream data according to the key frame slicing intervals to obtain a plurality of live broadcast fragments, and caching the live broadcast fragments, wherein the initial data frame of the video data corresponding to the live broadcast fragments is a key video frame;

receiving a download request sent by a playing terminal, taking a time point when the download request is received as a screening starting time point, screening the plurality of live broadcast fragments cached before the screening starting time point according to a screening rule to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set, wherein the screening rule comprises at least one of a rollback preset rollback time interval and a rollback preset screening number;

and according to the time point corresponding to each live broadcast fragment in the target live broadcast fragment set, sequentially sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit according to the sequence of the time point from back to front so as to enable the playing terminal to analyze the received live broadcast fragments, determining the number of live broadcast video frames corresponding to the received live broadcast fragments, sequentially accumulating the number of the live broadcast video frames in the sequence from back to front according to the time point of the live broadcast video frames analyzed from the live broadcast fragments, stopping receiving the live broadcast fragments when the accumulated number of the live broadcast video frames reaches a preset video frame number threshold value, taking the time point of the current live broadcast video frame of the current live broadcast fragment as a live broadcast point, and starting to play the received live broadcast fragments.

2. The method of claim 1, further comprising:

receiving live audio and video data and corresponding information data, wherein the live audio and video data comprises a plurality of live data blocks;

and acquiring the synchronization information corresponding to each frame of live video frame from the information data, and packaging the synchronization information to the live data block where the corresponding live video frame is located to form live streaming data.

3. The method of claim 1, wherein the screening the currently cached plurality of live broadcast segments according to the download request to obtain a target live broadcast segment set comprises:

acquiring a preset screening number, screening the live broadcast fragments with the preset screening number from the plurality of live broadcast fragments currently cached according to the downloading request to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set; and/or

Obtaining a preset backspacing time interval, and backspacing the preset backspacing time interval according to the downloading request to obtain a starting point of the starting live broadcast fragment;

and taking the start live broadcast segment corresponding to the start point of the start live broadcast segment and the live broadcast segment data after the start live broadcast segment as the target live broadcast segment, and forming the target live broadcast segment into the target live broadcast segment set.

4. The method of claim 1, wherein determining a key frame slicing interval according to the preset slicing interval and the key frame duration interval comprises:

when the ratio of the preset slicing time interval to the key frame time interval is an integer, multiplying the integer by the key frame time interval to obtain the key frame slicing interval;

and when the ratio of the preset slicing time interval to the key frame time interval is a non-integer, rounding up or rounding down the non-integer to obtain a target integer, and multiplying the target integer by the key frame time interval to obtain the key frame slicing interval.

5. The method of claim 1, further comprising:

the playing terminal analyzes the received live broadcast fragment to obtain the synchronization information corresponding to the live broadcast video frame in the received live broadcast fragment;

and synchronously playing or synchronously displaying the synchronization information corresponding to the live video frames in the received live video segments and the corresponding live video frames.

6. A live data processing system, the system comprising:

the processing server is used for receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data, the information data comprises at least one of title information, text content information, numerical value transfer information and live resource information required in a live broadcasting process, a preset slicing time interval is obtained, a key frame time interval corresponding to each video image group in the live streaming data is obtained, a key frame slicing interval is determined according to the preset slicing time interval and the key frame time interval, the preset slicing time interval is greater than the key frame time interval, and the key frame slicing interval is greater than the key frame time interval; slicing the live broadcast stream data according to the key frame slicing intervals to obtain a plurality of live broadcast fragments, and caching the live broadcast fragments, wherein the initial data frame of the video data corresponding to the live broadcast fragments is a key video frame;

the playing terminal is used for sending a downloading request to the processing server;

the processing server is further configured to receive a download request sent by the play terminal, filter the multiple live broadcast fragments cached before the screening start time point according to a filtering rule to obtain target live broadcast fragments, combine the target live broadcast fragments into a target live broadcast fragment set, and send data in the target live broadcast fragment set to the play terminal in units of the live broadcast fragments according to time points corresponding to the live broadcast fragments in the target live broadcast fragment set from the back to the front according to a sequence of the time points, where the filtering rule includes at least one of a rollback preset rollback time interval and a rollback preset filtering number;

according to the time point corresponding to each live broadcast fragment in the target live broadcast fragment set, the playing terminal is further used for sequentially receiving data in the target live broadcast fragment set by taking each live broadcast fragment as a unit according to the sequence of the time point from back to front; the playing terminal is further used for analyzing the received live broadcast fragments, determining the number of live broadcast video frames corresponding to the received live broadcast fragments, sequentially accumulating the number of the live broadcast video frames in a backward-forward sequence according to the time points of the live broadcast video frames analyzed from the live broadcast fragments, stopping receiving the live broadcast fragments when the accumulated number of the live broadcast video frames reaches a preset video frame number threshold value, taking the time points of the current live broadcast video frames of the current live broadcast fragments as live broadcast playing points, and starting playing the received live broadcast fragments.

7. The system of claim 6, further comprising a live server,

the live broadcast server is used for receiving live broadcast audio and video data and corresponding information data, wherein the live broadcast audio and video data comprises a plurality of live broadcast data blocks;

and the live broadcast server is also used for acquiring the synchronous information corresponding to each frame of live broadcast video frame from the information data and packaging the synchronous information to the live broadcast data block where the corresponding live broadcast video frame is positioned to form live broadcast stream data.

8. The system according to claim 6, wherein the processing server is further configured to obtain a preset screening number, and screen live broadcast segment data of the preset screening number from the currently cached multiple live broadcast segments according to the download request to obtain target live broadcast segment data; and/or

The processing server is further configured to obtain a preset back-off time interval, obtain an initial live broadcast segment starting point according to the download request back-off preset back-off time interval, and compose initial live broadcast segment data corresponding to the initial live broadcast segment starting point and live broadcast segment data subsequent to the initial live broadcast segment data into target live broadcast segment data.

9. The system according to claim 6, wherein the processing server is further configured to, when the ratio of the preset slicing time interval to the key frame duration interval is an integer, multiply the integer by the key frame duration interval to obtain the key frame slicing interval;

and the processing server is further configured to, when the ratio of the preset slicing time interval to the key frame time interval is a non-integer, round the non-integer up or down to obtain a target integer, and multiply the target integer by the key frame time interval to obtain the key frame slicing interval.

10. The system of claim 6, wherein the playing terminal is further configured to parse the received live broadcast segment to obtain synchronization information corresponding to a live broadcast video frame in the received live broadcast segment;

and synchronously playing or synchronously displaying the synchronization information corresponding to the live video frames in the received live video segments and the corresponding live video frames.

11. A live data processing apparatus, the apparatus comprising:

the live streaming data receiving module is used for receiving live streaming data, wherein the live streaming data comprises live audio and video data and corresponding information data, and the information data comprises at least one of title information, text content information, numerical value transfer information and live resource information required in a live broadcasting process;

the live streaming data slicing module is used for acquiring a preset slicing time interval, acquiring a key frame time interval corresponding to each video image group in live streaming data, determining a key frame slicing interval according to the preset slicing time interval and the key frame time interval, wherein the preset slicing time interval is greater than the key frame time interval, the key frame slicing interval is greater than the key frame time interval, slicing the live streaming data according to the key frame slicing interval to obtain a plurality of live streaming fragments, and caching the live streaming fragments, wherein the initial data frame of video data corresponding to the live streaming fragments is a key video frame;

a download request receiving module, configured to receive a download request sent by a play terminal;

the target live broadcast fragment screening module is used for taking the time point when the downloading request is received as a screening starting time point, screening the plurality of live broadcast fragments cached before the screening starting time point according to a screening rule to obtain target live broadcast fragments, and forming the target live broadcast fragments into a target live broadcast fragment set, wherein the screening rule comprises at least one of a backspacing preset backspacing time interval and a backspacing preset screening number;

and the live broadcast fragment sending module is used for sequentially sending the data in the target live broadcast fragment set to the playing terminal by taking each live broadcast fragment as a unit according to the time point corresponding to each live broadcast fragment in the target live broadcast fragment set from back to front according to the sequence of the time point, so that the playing terminal analyzes the received live broadcast fragments, determines the number of live broadcast video frames corresponding to the received live broadcast fragments, sequentially accumulates the number of the live broadcast video frames according to the time point of the live broadcast video frames analyzed from the live broadcast fragments and the sequence from back to front, stops receiving the live broadcast fragments when the accumulated number of the live broadcast video frames reaches a preset video frame number threshold value, and starts to play the received live broadcast fragments by taking the time point of the current video frame of the current live broadcast fragment as a live broadcast playing point.

12. The apparatus of claim 11, further comprising:

the live broadcast audio and video data receiving module is used for receiving live broadcast audio and video data and corresponding information data, wherein the live broadcast audio and video data comprises a plurality of live broadcast data blocks;

and the synchronous information acquisition module is used for acquiring the synchronous information corresponding to each frame of live video frame from the information data and packaging the synchronous information to the live data block where the corresponding live video frame is located to form live stream data.

13. The apparatus of claim 11, wherein the target live segment filtering module comprises:

a preset screening number obtaining unit for obtaining a preset screening number;

a target live broadcast fragment set generating unit, configured to screen, according to the download request, the live broadcast fragments of the preset screening number from the multiple live broadcast fragments currently cached to obtain target live broadcast fragments, and form the target live broadcast fragments into a target live broadcast fragment set; and/or

A preset rollback time interval obtaining unit, configured to obtain a preset rollback time interval, and obtain a starting point of an initial live broadcast segment according to the download request rollback preset rollback time interval;

and the target live broadcast segment set acquisition unit is used for taking the starting live broadcast segment corresponding to the starting point of the starting live broadcast segment and the live broadcast segment data after the starting live broadcast segment as the target live broadcast segment and forming the target live broadcast segment into the target live broadcast segment set.

14. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 5.

15. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 5.

Images