Millisecond, according to the mode ofstep 201, selecting one or several frames from the above N frames, extending the playing time of the live broadcast data of the selected frame (specifically how long it is extended, which can be flexibly set), if the selected frame is the kth frame in each first preset time length, extending the playing time of the frame by one time, then the original live broadcast data of M millisecond can be played

Milliseconds. If the first preset time duration M is 970 milliseconds, the number of frames N in the first preset time duration is 33 frames, that is, the playing time of each live broadcast is

Millisecond, selecting from the 33 frames, that is, the kth frame in each first preset time length is 3 frames, and extending the playing time of the frame by one time, so that the original 970 millisecond live broadcast data can be played

Milliseconds.

Compared with the mode of processing the playing speed of all live broadcast data in the prior art, firstly, the playing speed of most live broadcast data in the playing process is not adjusted, so that the playing speed of the live broadcast data perceived by a user when watching the live broadcast is real, and the optimal live broadcast effect and the optimal user perception are ensured. Secondly, the target frame is selected for delayed playing, the subsequent live broadcast data is cached, and the stable playing of the live broadcast data can be maintained.

Optionally, on the basis of fig. 1, the present application further provides a possible implementation manner of a live data processing method, and fig. 3 is a flowchart of a live data processing method provided in another embodiment of the present application, and as shown in fig. 3, if the first target live data is audio data, playing the first target live data at a first preset playing speed within a preset unit time duration includes:

step 301: and if the first target live broadcast data with the first preset duration comprises N frames, extracting two adjacent frames of the target according to a preset rule, wherein N is an integer greater than 0.

Similar to the extraction manner instep 201, one or more frames in the first target live broadcast data are extracted as two adjacent frames of the target according to a preset rule, which may be random extraction or a more detailed extraction rule, for example, the kth frame and the (k + 1) th frame in each first preset time period, where k is an integer between 1- (N-1). The application does not limit the specific number and the specific extraction mode of two adjacent frames of the target. In a possible implementation manner, for the audio data, the frame with the smallest difference from the audio data of the next frame to the target adjacent frame may be extracted as the target adjacent frame, for example, if the audio data of the kth frame and the (k + 1) th frame in a certain first preset time period are close (for example, the frequency, loudness, and the like of the audio data are close), the kth frame and the (k + 1) th frame may be selected as the target adjacent frame, so as to further reduce the influence of subsequent operations on the audio data on the intuitive experience of the user.

Step 302: and adding an insertion frame between two adjacent frames, and playing the first target live broadcast data, wherein the insertion frame is obtained by calculation according to a specified frame in the first target live broadcast data and a preset algorithm.

In a possible implementation manner, the first preset duration is M milliseconds, the number of frames in the first preset duration is N frames, two adjacent frames of the target are selected through theabove step 301, and an insertion frame is added between the two adjacent frames, so that the original audio data of M milliseconds can play (M + M/N) milliseconds. When calculating the specific data of the inserted frame, an averaging manner may be selected, for example, a specified frame in the first target live broadcast data is two adjacent frames to the target determined instep 301, and the specific data of the inserted frame is calculated by calculating an average value of audio frequency, loudness, and other data of the adjacent frames to the target, that is, a preset algorithm is the calculated average value. In a specific embodiment, if M equals 970 milliseconds and N equals 33 frames, a frame is inserted between the 3 rd frame and the fourth frame in each segment of the first preset duration, and the frequency loudness of the inserted sound frame is determined by taking an average value, so that the original 970 milliseconds of audio data can be played for (970+970/33) milliseconds.

The condition that the type of the live broadcast data is audio data is further limited, and the particularity of the audio data is considered, namely the audio data can be quantized, characteristic values such as frequency, loudness and the like after quantization can be calculated through a preset algorithm, and the calculation result also has use value. And carrying out interpolation processing on the audio data, and carrying out processing calculation on the inserted frame according to a preset algorithm so as to ensure the quality of the audio data received and heard by the user and the quality of the audio data.

Optionally, on the basis of fig. 1, the present application further provides a possible implementation manner of a live data processing method, and fig. 4 is a flowchart of a live data processing method according to yet another embodiment of the present application; as shown in fig. 4, the method includes:

step 401: and acquiring second live broadcast data sent by the server.

And the live broadcast client acquires second live broadcast data sent by the server, namely the live broadcast client downloads the second live broadcast data from the server. It should be noted that the second live data and the first live data obtained instep 101 may be continuous in time, that is, the second live data and the first live data can be played continuously; the second live data may also be discontinuous in time with the first live data acquired instep 101, because a part of the data after the first preset time duration has been cached instep 103, so that a section of live data cached instep 103 still exists between the first live data and the second live data.

Step 402: playing second target live broadcast data within a preset unit time length at a first preset playing speed, wherein the second target live broadcast data comprise: the buffered data and the data of the previous part of the duration in the second live broadcast data, and the total duration of the buffered data and the duration of the part of the duration is a first preset duration.

Playing the second target live broadcast data at a first preset playing speed which is the same as the playing speed instep 102 within a preset unit time length, where the second target live broadcast data may include: the cached data may be a part of live broadcast data cached instep 103, or live broadcast data cached in other caching manners, and the caching manner of the cached data is not limited in the present application. The total duration of the duration and the partial duration of the buffered data is a first preset duration, that is, the sum of the duration of the buffered data and the duration of the previous partial duration (that is, the partial duration) in the second live broadcast data is the first preset duration.

In a specific implementation manner, the preset unit time length is P milliseconds, the first preset time length is M milliseconds, and the time length of the cached data instep 103 is P-M milliseconds, so that the previous M- (P-M) milliseconds of live broadcast data in the second live broadcast data is needed, and the second target live broadcast data is obtained after the connection between the previous M- (P-M) milliseconds of live broadcast data and the cached P-M milliseconds of data according to the playing sequence. The processing method and the playing method of the second target live broadcast data are the same as those instep 102, and are not described herein again. For example, P equals 1000 ms, M equals 970 ms, P-M equals 30 ms, and M- (P-M) equals 940 ms.

Step 403: and caching the data after the played data in the second live data.

As can be seen fromstep 402, a time difference exists between the preset unit time length and the previous part time length of the second live data in the second target live data, that is, the time length of the live data played in the preset unit time length and the time length of the live data that needs to be acquired from the server, and the live client can cache the data after the played data in the second live data by using the time difference between the preset unit time length and the previous part time length of the second live data in the second target live data.

In a specific implementation manner, the preset unit time length is P milliseconds, the first preset time length is M milliseconds, and the time length of the cached data is D₀If the time length of the live broadcast data needing to be acquired from the server in the live broadcast data played in the preset unit time length is M-D, the time length of the live broadcast data needing to be acquired from the server is M-D₀The number of milliseconds (ms),the time difference between the time length of the live broadcast data played in the preset unit time length and the time length of the live broadcast data needing to be acquired from the server is P- (M-D)₀) The P- (M-D) after the played data in the second live broadcast data can be cached in millisecond₀) Milliseconds of live data. E.g., P1000 ms, M970 ms, D₀90 ms, M-D₀970-90 ms, P- (M-D)₀) If 1000- (970-90) ═ 120 ms, the second live data can be buffered 120 ms after the played data.

The above is merely an example, in an actual application process, live broadcast data after playing data in the second live broadcast data is cached under the influence of a current network state and the like may not conform to a theoretical or ideal condition, and the specific time length of the cached live broadcast data in a specific implementation process is not limited in the present application.

If the server is used for caching the live audio and video data of several seconds, the client receives the data of several seconds from the beginning and caches the data locally, so that the network jitter situation can be resisted, but the several seconds of audio and video data cached by the server are all before the current time, so that the content seen by the live client is delayed for a large amount and is not real-time enough, and even if the network is good, the client also has several seconds of delay. In the method, the data after the data is played in the live broadcast data is continuously cached in the live broadcast process, so that the caching duration is gradually prolonged, the stable playing of the live broadcast data is further maintained, and the delay is obviously improved compared with the method.

Optionally, on the basis of fig. 4, the present application further provides a possible implementation manner of a live data processing method, and fig. 5 is a flowchart of a live data processing method according to a second embodiment of the present application, where as shown in fig. 5, before data after playing data in second live data is cached, the method includes:

step 501: and if the network state between the server and the server is a first network state, determining a first preset time threshold according to the network state, wherein the first preset time threshold is smaller than a first preset time.

It should be noted that there are many possible implementation manners for judging the network state between the live broadcast client and the server, and in one possible implementation manner, the transmission speed of the live broadcast data can be influenced by the quality of the network state, and the transmission speed of the live broadcast data can influence the speed of the live broadcast data in the cache in an access alternation manner, so that the network jitter estimated value of the initial live broadcast data and the network jitter estimated value of the real-time live broadcast data can be compared to determine the network state.

For example: when the network state is good, the alternating speed of the live broadcast data access is high, the buffered live broadcast data is in accordance with expectation, the smaller the network jitter estimated value is, when the network state is poor, the lower the alternating speed of the live broadcast data access is, the lower the buffer is in accordance with the expectation, and the larger the network jitter estimated value is. Therefore, the network state can be judged according to the network jitter estimated values respectively obtained by the initial live broadcast data and the real-time live broadcast data.

In another possible implementation manner, the network state may be evaluated by determining the current network delay, for example, a network delay determination criterion corresponding to a general network state, a good network state, a bad network state, and the like is preset, and the current network state is determined according to a network delay value obtained by live broadcasting and the preset network delay determination criterion.

For example, it is generally assumed that the network delay is set to be in a network state between a ms and B ms in advance, and the network state is good when the network delay is less than a ms and not good when the network delay is greater than B ms. If a is 100 ms and B is 200 ms, the network state is generally determined when the network delay is between 100 ms and 200 ms, and the network state is good when the network delay is less than 100 ms and not good when the network delay is greater than 200 ms.

The foregoing is merely an example, and in practical applications, there may be other methods for determining a network state, and the specific method for determining a network state is not limited in this application.

In a specific implementation manner, a network delay value obtained by live broadcasting is compared with a preset network delay judgment standard, the network delay is set to be in a first network state between a millisecond and B millisecond, and a first preset duration threshold in the first network state is set to be D1 millisecond. In one possible implementation, a is 100 ms, B is 200 ms, and D1 is 600 ms. It should be noted that, the specific value of the first preset duration threshold is not limited in the present application, as long as it meets the condition that it is less than the first preset duration.

Caching the data after the played data in the second live data, including:

step 502: and caching the data after the data is played in the second live broadcast data until the time length of the cached data reaches a first preset time length threshold value.

According to step 403, the live broadcast client may utilize a time difference between a time length of live broadcast data played in a preset unit time length and a time length of live broadcast data that needs to be acquired from the server, to cache data after the data has been played in the second live broadcast data, after the caching instep 403 is performed for a plurality of times, the time length of the cached live broadcast data is continuously increased, and when the time length of the cached live broadcast data reaches a first preset time length threshold, the time length of the cached live broadcast data is not continuously increased, but the state that the time length of the cached live broadcast data is equal to the first preset time length threshold is maintained.

It should be noted that, in a possible implementation manner, when the duration of the cached live data reaches a first preset duration threshold, the duration of the live data played on the live client is equal to the duration preset by the first target live data itself downloaded from the server. For example, for live data with a first preset time duration of P milliseconds from the server, the live client consumes M milliseconds (P ═ M) of the objective world when playing on the live client, in other words, all the live data with P milliseconds played by the live client comes from the server, and the live client does not process the live data any more, so that the delay of the live data on the live client is equal to the first preset time duration threshold and the state is maintained unchanged. In one specific implementation, P-M-970 milliseconds. The above is merely an example, and when the cached data time length reaches the first preset time length threshold, there may be other methods for maintaining the cached data time length, which is not limited in the present application.

The buffer time is too long, which causes the delay of the live broadcast data to be larger, and the buffer time is too short, which is not favorable for the stability of the live broadcast, and the situation of blocking or fuzzy picture can occur under the conditions of network jitter, etc., therefore, the time length of the buffered live broadcast data is limited according to the network state, which not only ensures the stable broadcast in the current network environment, but also can maintain the broadcast effect with the delay as low as possible.

Optionally, on the basis of fig. 5, the present application further provides a possible implementation manner of a live data processing method, and fig. 6 is a flowchart of a live data processing method according to a further third embodiment of the present application; as shown in fig. 6, the method includes:

step 601: and if the network state between the server and the server is a second network state, determining a second preset time length threshold according to the network state, wherein the delay of the second network state is greater than the delay of the first network state, and the second preset time length threshold is greater than the preset unit time length.

Because the delay of the second network state is greater than the delay of the first network state, the live broadcast data is stably played in the second network state, and more live broadcast data needs to be cached, so the cached data time length is set to be a second preset time length threshold value, and the second preset time length threshold value is greater than the first preset time length threshold value in the first network state.

In a specific implementation manner, a second network state is set when the network delay is greater than B milliseconds by comparing a network delay value obtained by live broadcasting with a preset network delay judgment standard, and a second preset duration threshold in the second network state is set to D2 milliseconds. For example, if B is 200, D2 is 1200 ms. It should be noted that, the specific value of the second preset duration threshold is not limited in the present application, as long as it meets the condition that the second preset duration is less than the second preset duration.

Step 602: and playing the live broadcast data received from the server at a first preset playing speed, and caching the data after the played data in the received live broadcast data until the time length of the cached data reaches a second preset time length threshold value.

The first preset playing speed is the playing speed instep 102, the live data received from the server is played at this speed, the data after the data has been played in the live data is continuously cached by using the time difference between the duration of the live data played in the preset unit time length and the duration of the live data needing to be acquired from the server, when the duration of the cached live data reaches the second preset time length threshold, the duration of the cached live data is not continuously increased, but the duration of the cached live data is maintained to be equal to the second preset time length threshold.

It should be noted that, in a possible implementation manner, when the duration of the cached live data reaches a second preset duration threshold, the duration of the live data played on the live client is equal to the preset duration of the second target live data downloaded from the server. For example, for live broadcast data with a second preset time length of L milliseconds from the server, the live broadcast client consumes Q milliseconds (L ═ Q) of the objective world when playing on the live broadcast client, in other words, all the live broadcast data with Q milliseconds played by the live broadcast client comes from the server, and the live broadcast client does not process the live broadcast data any more, so that the delay of the live broadcast data on the live broadcast client is equal to the second preset time length threshold, and the state is maintained unchanged. For example, L ═ Q ═ 970 milliseconds. The above is merely an example, and when the cached data time length reaches the second preset time length threshold, there may be other methods for maintaining the cached data time length, which is not limited in the present application.

The quality of the network affects the transmission of live broadcast data, and further affects the stability of live broadcast, when the network is not good, the transmission of the live broadcast data becomes slow, and the network jitter can be resisted by caching more live broadcast data, so that the stability of the live broadcast is further enhanced.

Optionally, on the basis of fig. 6, the present application further provides a possible implementation manner of a live data processing method, and fig. 7 is a flowchart of a live data processing method according to yet another fourth embodiment of the present application; as shown in fig. 7, the method includes:

step 701: and if the network state is switched from the second network state to a third network state, determining a third preset time threshold according to the network state, wherein the delay of the third network state is less than that of the first network state, and the third preset time threshold is less than the first preset time threshold.

Because the delay of the third network state is less than the delay of the first network state, in the third network state, the excessive live broadcast data does not need to be cached, and therefore the cached data time length is set to be a third preset time length threshold value, and the third preset time length threshold value is less than the first preset time length threshold value in the first network state.

In a specific implementation manner, a third network state is set when the network delay is less than a millisecond by comparing a network delay value obtained by live broadcasting with a preset network delay judgment standard, and a third preset duration threshold in the third network state is set to be D3 milliseconds. For example, if a is 100 ms, D3 is 400 ms. It should be noted that, in the present application, a specific value of the third preset duration threshold is not limited.

Step 702: and playing the live broadcast data received from the server at a second preset playing speed, and caching the data after the played data in the received live broadcast data until the time length of the cached data reaches a third preset time length threshold value.

If the current network state is the third network state, if the duration of the currently cached live broadcast data is greater than a third preset duration threshold, the cached live broadcast data needs to be shortened by fast playing, that is, the live broadcast data received from the server is played at the second preset playing speed. The second preset playing speed is greater than the first preset playing speed, and the second preset playing speed is also greater than the preset playing speed of the first target live broadcast data downloaded from the server.

Namely, the second playing speed of the live broadcast client represents that the time consumed by the live broadcast data on the live broadcast client is shorter, namely, the second preset playing speed is greater than the preset playing speed for downloading the live broadcast data from the server. In a specific implementation manner, live data is downloaded from a server within a certain time, and M millisecond data is preset and played, and in the same time, a live client can play F millisecond live data, that is, the second play speed indicates the speed at which the live client plays the F millisecond live data within the certain time, so that fast play is realized, and then cache is reduced until the duration of the cached data is equal to a third preset duration threshold. For example, M1000 ms and F1030 ms.

When the network state is good, the state that the playing delay of the live client is as small as possible can be realized by reducing the cached live data.

Optionally, on the basis of the foregoing embodiments, the present application further provides a possible implementation manner of a live data processing method, where the method includes:

and if the live broadcast mode is the low-delay mode, playing the first target live broadcast data within the preset unit time length at a first preset playing speed.

The method includes the steps that a plurality of live broadcast playing modes can be available on a live broadcast client, for example, the live broadcast playing modes can include a smooth mode, a high-definition mode, a low-delay mode and the like, a user can select the specific live broadcast playing mode on the live broadcast client, and if the user selects the low-delay mode of the application, first target live broadcast data are played at a first preset playing speed of the live broadcast client within preset unit time. It should be noted that, the present application is not limited to specific types of other live play modes included in the live client.

Specific implementation processes and technical effects of the following descriptions of the live data processing apparatus, the live client device, the storage medium, and the like for executing the live data processing apparatus, the live client device, and the storage medium provided by the present application are referred to above, and are not described in detail below.

Embodiments of the present application provide possible implementation examples of a live data processing apparatus, which can execute the live data processing method provided in the foregoing embodiments. Fig. 8 is a schematic diagram of a live data processing apparatus according to an embodiment of the present application. As shown in fig. 8, the livedata processing apparatus 100 includes: a receivingmodule 601, aplaying module 603 and abuffer module 605.

A receivingmodule 601, configured to obtain first direct broadcast data sent by a server;

theplaying module 603 is configured to play the first target live broadcast data at a first preset playing speed within a preset unit duration, where the first target live broadcast data is: the preset unit time length of the data with the first preset time length in the first direct broadcasting data is greater than the first preset time length;

thecaching module 605 is configured to cache data in the first direct broadcast data after a first preset time duration.

Optionally, the receivingmodule 601 is further configured to obtain second live data sent by the server;

theplaying module 603 is further configured to play the second target live broadcast data at a first preset playing speed within a preset unit time duration, where the second target live broadcast data includes: the cached data and the data of the previous part of the duration in the second live broadcast data, wherein the duration of the cached data and the total duration of the part of the duration are first preset duration;

thecaching module 605 is further configured to cache data after the played data in the second live data.

Optionally, the live data processing apparatus further includes: and a network state judgment module.

And the network state judging module is used for judging the network state, and if the network state between the server and the network state judging module is the first network state, determining a first preset time length threshold value according to the network state, wherein the first preset time length threshold value is smaller than the first preset time length.

Thecaching module 605 is specifically configured to cache data after the data has been played in the second live data until the duration of the cached data reaches a first preset duration threshold.

Optionally, the network state determining module is configured to determine a network state, and determine a second preset time threshold according to the network state if the network state between the server and the network state is a second network state, where a delay of the second network state is greater than a delay of the first network state, and the second preset time threshold is greater than a preset unit time.

The receivingmodule 601 is specifically configured to play the live data received from the server at a first preset playing speed.

Thecaching module 605 is specifically configured to cache data after the data has been played in the received live data until the duration of the cached data reaches a second preset duration threshold.

Optionally, the network state determining module is configured to determine a network state, and determine a third preset duration threshold according to the network state if the network state is switched from the second network state to a third network state, where a delay of the third network state is smaller than a delay of the first network state, and the third preset duration threshold is smaller than the first preset duration threshold.

The receivingmodule 601 is specifically configured to play the live data received from the server at a second preset playing speed.

Thecaching module 605 is specifically configured to cache data after the data has been played in the received live data until the duration of the cached data reaches a third preset duration threshold.

Optionally, the live data processing apparatus further includes: and a mode selection module.

And the mode selection module is specifically used for selecting a live broadcast playing mode, and if the live broadcast playing mode is a low-delay mode, playing the first target live broadcast data within a preset unit time length at a first preset playing speed.

The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.

These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

The embodiment of the present application provides a possible implementation example of a live client device, which can execute the live data processing method provided by the foregoing embodiment. Fig. 9 is a schematic diagram of a live client device according to an embodiment of the present application, where the device may be integrated in a terminal device or a chip of the terminal device, and the terminal may be a computing device with a data processing function.

The live client device includes: the system comprises aprocessor 701, astorage medium 702 and a bus, wherein the storage medium stores program instructions executable by the processor, when the control device runs, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to execute the steps of the live broadcast data processing method. The specific implementation and technical effects are similar, and are not described herein again.

The embodiments of the present application provide possible implementation examples of a computer-readable storage medium, which is capable of executing the live data processing method provided in the foregoing embodiments, where the storage medium stores a computer program, and the computer program is executed by a processor to execute the steps of the live data processing method.

A computer program stored in a storage medium may include instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor (which may be a processor) to perform some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other media capable of storing program codes.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.