CN111818279A

Movatterモバイル変換

Info

Publication number: CN111818279A
Application number: CN201910294728.2A
Authority: CN
Inventors: 彭俊; 张磊
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-04-12
Filing date: 2019-04-12
Publication date: 2020-10-23

Abstract

The invention discloses a subtitle generating method, a subtitle displaying method and an interaction method, wherein the subtitle generating method comprises the following steps: acquiring a standard subtitle file of a video, wherein the standard subtitle file comprises subtitle content; processing the standard subtitle file with an additional function to obtain a compound subtitle file, wherein the compound subtitle file comprises the subtitle content and additional information content corresponding to the additional function; and sending the compound caption file to a terminal device for displaying the compound caption file.

Description

Subtitle generating method, display method and interaction method

Technical Field

The present invention relates to the technical field of subtitle generation, and more particularly, to a subtitle generation method, a subtitle display method, a subtitle interaction method, a server, a terminal device, and a subtitle generation system.

Background

With the development of internet technology, more and more users watch videos on terminal devices, wherein the videos are displayed with subtitles so as to facilitate better understanding of the contents of the videos.

In the video playing process, the subtitle has a single function and is only displayed in a fixed mode.

Disclosure of Invention

It is an object of the present invention to provide a new solution for generating subtitles for video display.

According to a first aspect of the present invention, there is provided a method for generating a subtitle, implemented by a server, including:

acquiring a standard subtitle file of a video, wherein the standard subtitle file comprises subtitle content;

processing the standard subtitle file with an additional function to obtain a compound subtitle file, wherein the compound subtitle file comprises the subtitle content and additional information content corresponding to the additional function;

and sending the compound caption file to terminal equipment for displaying the compound caption file.

Optionally, the method further comprises:

detecting whether an instruction for processing an additional function on the standard subtitle file exists or not to obtain a detection result;

under the condition that the detection result is negative, the standard subtitle file is directly sent to the terminal equipment;

and if the detection result is yes, executing the processing of performing the additional function on the standard subtitle file to obtain the operation of the compound subtitle file.

Optionally, the obtaining the standard subtitle file of the video includes:

performing voice recognition processing on the video to obtain subtitle content;

and generating a standard subtitle file of the video by using the subtitle content.

Optionally, the additional function includes at least one of a subtitle change function and a subtitle link function.

Optionally, the subtitle change function includes at least one of a subtitle display position change function, a subtitle display color change function, and a subtitle display font change function.

Optionally, the subtitle link function includes at least one of a function of implementing a jump to another page through a subtitle link, a function of implementing content corresponding to an attachment displayed on a current page through a subtitle link, and a function of adding a background layer to a subtitle through a subtitle link.

According to a second aspect of the present invention, there is provided a method for generating subtitles, implemented by a terminal device, comprising:

receiving a composite caption file generated according to any one of the methods in the first aspect and sent by a server;

analyzing the compound caption file to obtain caption content and additional information content corresponding to additional functions;

and rendering the subtitle content and the additional information content corresponding to the additional function to obtain a composite subtitle displayed in the video so that the composite subtitle has the additional function.

Optionally, the additional function includes at least one of implementing a subtitle change function and implementing a subtitle link function.

Optionally, the additional information content includes link information for implementing a subtitle link function, wherein the method further includes:

forming interactive information for realizing human-computer interaction according to the link information;

responding to the click operation of the compound caption, and detecting whether the click operation is performed according to the interactive information;

and providing the linked content according to the interactive information under the condition that whether the clicking operation is performed according to the interactive information.

According to a third aspect of the present invention, there is provided a method for displaying a subtitle, comprising:

acquiring a compound subtitle file from a server, wherein the compound subtitle file comprises subtitle content and additional information content for realizing a subtitle change function;

reading the subtitle content in the compound subtitle file and the additional information content for realizing the subtitle change function;

and generating and displaying a composite subtitle with a subtitle change function based on the subtitle content and additional information content for realizing the subtitle change function, wherein the display effect of the composite subtitle has at least one of subtitle display position change, subtitle display color change and subtitle display font change.

According to a fourth aspect of the present invention, there is provided a subtitle interaction method, including:

acquiring a compound subtitle file from a server, wherein the compound subtitle file comprises subtitle content and additional information content for realizing a subtitle link function;

reading the subtitle content in the compound subtitle file and the additional information content for realizing the subtitle link function;

generating and displaying a composite subtitle with a subtitle linking function based on the subtitle content and additional information content for realizing the subtitle linking function;

and executing the subtitle link function in response to the interactive instruction aiming at the compound subtitle.

According to a fifth aspect of the present invention, there is provided a server comprising a memory for storing executable instructions and a processor; the processor is configured to perform the method according to any one of the first aspect under control of the instructions.

According to a sixth aspect of the present invention, there is provided a terminal device comprising a memory for storing executable instructions and a processor; the processor is configured to perform the method according to any one of the second aspects under control of the instructions.

According to a seventh aspect of the present invention, there is provided a subtitle generating system comprising the server of the fifth aspect and the terminal device of the sixth aspect.

The invention has the advantages that the invention realizes the multi-functionalization of the caption and no longer only provides a display function.

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a schematic diagram of subtitles displayed when a terminal device plays a video in the prior art.

Fig. 2a is a schematic diagram of a composition structure of a subtitle generating system supporting implementation of the subtitle generating method according to an embodiment of the present invention.

Fig. 2b is a schematic diagram of a hardware structure of a server in the system for generating subtitles according to the embodiment of the present invention.

Fig. 3 is a schematic block diagram illustrating a structure of a subtitle generating system according to an embodiment of the present invention.

Fig. 4a is a flowchart illustrating a subtitle generating method according to an embodiment of the present invention.

Fig. 4b is a schematic illustration of video pictures corresponding to different time instants based on the example provided in fig. 4 a.

Fig. 5a is a flowchart illustrating a subtitle generating method according to another embodiment of the present invention.

Fig. 5b is a schematic illustration of a video picture based on the example provided in fig. 5 a.

Fig. 6 is a process flow diagram of a subtitle generating method performed by a server according to one embodiment of the present invention.

Fig. 7 is a process flow diagram of a subtitle generating method performed by a terminal device according to one embodiment of the present invention.

Fig. 8 is a flowchart illustrating a subtitle presentation method according to an embodiment of the present invention.

Fig. 9 is a flowchart illustrating an interactive method of subtitles according to an embodiment of the present invention.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

< hardware configuration >

In order to facilitate users to better understand video content, the video playing platform may configure a corresponding subtitle file for the video file, so that the terminal device can display the corresponding subtitle when playing the video. For example, terminal device 2000 shown in fig. 1 is playing a video about street dance, and the subtitle displays "this is street dance" at this time.

In the prior art, the subtitle "this is street dance" shown with reference to fig. 1, which is a standard subtitle 3000 generated based on a standard subtitle protocol. The standard caption only has a character display function and a single function. In order to solve the problem of single function of the existing subtitles, an embodiment of the present invention provides a new method for generating subtitles to add other functions to the subtitles besides the presentation function, for example, a subtitle change function, a subtitle link function, and the like.

The subtitle change function includes, for example, a function of realizing a change in the subtitle itself, such as a change in the subtitle display position (i.e., a subtitle movement function), a change in the subtitle display color, and a change in the subtitle display font. Since the subtitle change information belongs to subtitle attribute information for defining a subtitle format, in an embodiment of the present invention, additional information content corresponding to implementing a subtitle change function is also referred to as meta-subtitle protocol content.

The subtitle link function includes, for example, a function of jumping to another page through a subtitle link, a function of displaying content corresponding to an attachment on a current page through the subtitle link, and a function of adding a background layer to a subtitle through the subtitle link, that is, a function of providing a human-computer interaction. In the embodiment of the present invention, the additional information content corresponding to the additional linking function is also referred to as extended caption protocol content. In the above subtitle linking function, skipping means: the user clicks the link provided by the extended caption and can jump to another page; the accessories are as follows: the user clicks a link provided by the extended caption, and a small window or a dialog box is displayed on the current page to display the content corresponding to the attachment; the superposition means that: and clicking a link provided by the extended caption by the user, and adding a background layer for the extended caption, wherein the background layer can be a picture corresponding to the link.

Fig. 2a is a schematic structural diagram of a subtitle generating system supporting implementation of the subtitle generating method according to an embodiment of the present invention.

According to fig. 2a, the system for generating subtitles includes a server 1000 and a terminal device 2000, and the server 1000 and the terminal device 2000 are connected through network communication to realize data transmission.

In the generation system, the server 1000 pushes a video to the terminal device 2000, and the terminal device 2000 plays the video. The video may be a live video, an on-demand video, or the like, and is not limited herein.

The server 1000 provides a service point for processes, databases, and communications facilities. The server 1000 may be a unitary server or a distributed server across multiple computers or computer data centers. The server may be of various types, such as, but not limited to, a web server, a news server, a mail server, a message server, an advertisement server, a file server, an application server, an interaction server, a database server, or a proxy server. In some embodiments, each server may include hardware, software, or embedded logic components or a combination of two or more such components for performing the appropriate functions supported or implemented by the server. For example, a server, such as a blade server, a cloud server, etc., or may be a server group consisting of a plurality of servers, which may include one or more of the above types of servers, etc.

In one example, the server 1000 may be as shown in fig. 2a, including aprocessor 1100, a memory 1200, aninterface device 1300, acommunication device 1400, adisplay device 1500, aninput device 1600. The server may also include speakers, microphones, and the like.

Theprocessor 1100 may be, for example, a central processing unit CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. Theinterface device 1300 includes, for example, a USB interface, a serial interface, an infrared interface, and the like.Communication device 1400 is capable of wired or wireless communication, for example. Thedisplay device 1500 is, for example, a liquid crystal display, an LED display touch panel, or the like. Theinput device 1600 may include, for example, a touch screen, a keyboard, and the like.

Although a plurality of devices are shown for the server 1000 in fig. 2a, the present invention may relate to only some of the devices, for example, the server 1000 only relates to the memory 1200 and theprocessor 1100, and the terminal device 2000 only relates to thememory 2200 and theprocessor 2100.

In one example, the server 1000 may be a group of servers deployed in the cloud. As shown in fig. 2b, the server group may include only theprocessor 1100 and the memory 1200 to be operated by theprocessor 1100 in the server cluster to perform the subtitle generating method according to any embodiment of the present invention.

The terminal device 2000 may be a mobile phone, a laptop, a tablet computer, a palmtop computer, or other electronic devices.

The terminal device 2000 may include aprocessor 2100, amemory 2200, an interface device 2300, acommunication device 2400, adisplay device 2500, an input device 2600, a speaker 2700, a microphone 2800, a subtitle processing device 2900, and the like. Theprocessor 2100 may be a central processing unit CPU, a microprocessor MCU, or the like. Thememory 2200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. The interface device 2300 includes, for example, a USB interface, a headphone interface, and the like.Communication device 2400 is capable of wired or wireless communication, for example. Thedisplay device 2500 is, for example, a liquid crystal display panel, a touch panel, or the like. The input device 2600 may include, for example, a touch screen, a keyboard, and the like. A user can input/output voice information through the speaker 2700 and the microphone 2800.

The terminal device 2000 and the server 1000 are communicatively connected via a network. The network may be a wireless communication network or a wired communication network, and may be a local area network or a wide area network.

Although fig. 2a only shows one server 1000 and one terminal device 2000, it should be understood that, in a specific application, the live broadcast system may include a plurality of servers 1000 and a plurality of terminal devices 2000 according to actual requirements.

In an embodiment of the present invention, the memory 1200 of the server 1000 is configured to store instructions for controlling theprocessor 1100 to execute the method for generating subtitles according to the embodiment of the present invention. Thememory 2200 of the terminal device 2000 is configured to store instructions for controlling theprocessor 2100 to execute the method for generating a subtitle according to the embodiment of the present invention.

In the above description, the skilled person will be able to design instructions in accordance with the disclosed solution. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.

Fig. 3 is a schematic structural block diagram of a subtitle generating system according to an embodiment of the present invention, which is configured to implement a subtitle generating method according to an embodiment of the present invention to configure a compound subtitle file with additional functions for a video file, where the compound subtitle file includes text content carried by a standard subtitle file and additional information content for adding other additional functions.

As shown in fig. 3, the subtitle generating system includes a server 1000 and a terminal device 2000. Theprocessor 1100 of the server 1000 has a subtitle generating module including, for example, a standard subtitle file generating module 1110 and a compound subtitle file generating module 1120, so that theprocessor 1100 can perform a method of generating a subtitle according to an embodiment of the present invention through the subtitle generating module.

The standard subtitle file generating module 1110 is configured to process the received video file to obtain a standard subtitle file. The standard caption file comprises text content obtained by identifying and processing audio data carried by the video file.

When there is no instruction to add additional information content to the text content carried by the standard subtitle file, the standard subtitle file generating module 1110 is further configured to input the generated standard subtitle file to the terminal device 2000.

When there is an instruction to add additional information content to the text content carried by the standard subtitle file, the standard subtitle file generating module 1110 is further configured to input the generated standard subtitle file to the compound subtitle file generating module 1120.

The standard subtitle file generating module 1110 includes a standard subtitle interface unit 1111 and a standard subtitle file generating unit 1112.

The standard subtitle interface unit 1111 is configured to receive a video file.

The standard subtitle file generating unit 1112 is configured to generate a corresponding standard subtitle file from the video file. After generating the standard subtitle file, the standard subtitle file generating unit 1112 is further configured to input the standard subtitle file to the terminal device 2000 or the compound subtitle file generating module 1120.

The compound subtitle file generating module 1120 is configured to receive the standard subtitle file provided by the standard subtitle file generating module 1110, and perform additional function processing on the standard subtitle file to obtain a compound subtitle file. The compound subtitle file includes text content and additional information content carried by the standard subtitle file. The additional information content may include at least one of a function to implement a subtitle change function and link information to implement a subtitle link function. The additional information content may be divided into meta-subtitle protocol content and extended-subtitle protocol content according to the case of the additional information content.

The subtitle file generating module 1120 includes a subtitle interface unit 1121 and a subtitle file generating unit 1122.

The composite caption interface unit 1121 is configured to receive the standard caption file sent by the standard caption interface unit 1111.

The compound subtitle file generating unit 1122 is configured to perform additional function processing on the standard subtitle file to obtain a compound subtitle file. After generating the compound caption file, the compound caption file generating unit 1122 is also configured to provide the compound caption file to the terminal device 2000.

Theprocessor 2100 of the terminal device 2000 may include a subtitle processing module for processing subtitles, which may include a subtitle engine module 2110 and asubtitle rendering module 2120 as shown in fig. 3, so that theprocessor 2100 can perform a generation method of subtitles according to an embodiment of the present invention through the subtitle processing module.

The subtitle engine module 2110 includes a subtitle parsing interface unit 2111, a parsing unit 2112, and an extension unit 2113.

The subtitle parsing interface unit 2111 is configured to receive the standard subtitle file provided by the standard subtitle file generating unit 1112 or receive the compound subtitle file sent by the compound subtitle file generating unit 1122.

The parsing unit 2112 is configured to parse the standard subtitle file to obtain subtitle content; or analyzing the compound caption file to obtain the caption content and the additional information content.

The extension unit 2113 is configured to, in a case that the additional information content includes link information, throw the link information to the interaction processing module 2130 of the interaction layer (UI layer), so that the interaction processing module 2130 can sense an action of the user clicking an extended subtitle based on the link information, and implement a corresponding link function in response to the action. The link information may include, for example, a link address and an area where the link is located.

The interaction processing module 2130 is located on the UI layer, and is an entrance for the user to perform human-computer interaction, and all functions provided by the video interaction platform are provided to the user through the layer.

It should be apparent to those skilled in the art that for an application scenario in which a compound caption file including meta caption protocol content is provided but a compound caption file including extended caption protocol content is not provided, the caption engine module 2110 of the terminal device 2000 may not need to be provided with the extension unit 2113 for interacting with an upper layer.

Thesubtitle rendering module 2120 includes a subtitle rendering interface unit 2121 and a rendering unit 2122.

The subtitle rendering interface unit 2121 is configured to receive subtitle content; or for receiving subtitle content and additional information content.

The rendering unit 2122 is configured to render the subtitle content, or render the subtitle content and the additional information content, to obtain a subtitle displayed in the video.

In this embodiment, the rendering unit may employ different rendering engines to perform rendering operations, such as D3D (Direct3D), opengl (open Graphics library), gdi (Graphics Device interface), and Native.

< example 1>

Based on the subtitle generating system provided by the embodiment of the present invention, a specific process of subtitle generation is described below by taking the example of forming a composite subtitle including meta subtitle protocol content. Fig. 4a shows a flowchart of the subtitle generating method of this example.

In this example, according to the illustration in fig. 4a, on the side of the server 1000, the standard caption file generation module 1110 is configured to receive a video file and generate a corresponding standard caption file according to the video file. The standard subtitle file includes, for example, subtitle content of "00: 00:15.000- >00:00:18.000, which is street dance". The caption content is displayed at the caption display position during the 15 th to 18 th seconds after the start of the video, which is "street dance".

After the standard subtitle file generating module 1110 obtains the standard subtitle file, the standard subtitle file may be input to the compound subtitle file generating module 1120.

The compound subtitle file generating module 1120 is configured to receive the standard subtitle file provided by the standard subtitle file generating module 1110, and perform additional function processing on the standard subtitle file to obtain a compound subtitle file. In this example, the compound subtitle file includes text content and meta subtitle protocol content carried by a standard subtitle file. For example, the compound caption file is 00:00:15.000 to >00:00:18.000[ cirlcce (1000,480,16,3000) ], which is [ street dance ], wherein the meta caption protocol content is [ cirlcce (1000,480,16,3000) ]. The [ cirlce (1000,480,16,3000) ] is a function of the motion of the subtitle content, which means that the street dance makes a 16-radius rotation around the center point (1000, 480) with a duration of 3000 ms.

After the compound subtitle file generating module 1120 of the server 1000 generates the compound subtitle file, the compound subtitle file generating module 1120 transmits the compound subtitle file to the terminal device 2000.

As shown in fig. 4a, on the side of the terminal device 2000, after receiving the compound subtitle file, the subtitle engine module 2110 parses the compound subtitle file, and obtains the subtitle content and the display position of the subtitle content at each time. In this example, based on the content of the meta-caption protocol, i.e., [ circe (1000,480,16,3000) ] function, the display position of the street dance at each time is obtained through analysis, i.e., the display position of the street dance is (1000,496) in the time period 00:00:15.000- - >00:00:15.600, the display position of the street dance is (984,480) in the time period 00:00:15.600- - >00:00:16.200, the display position of the street dance is (1000,466) in the time period 00:00:16.200- - >00:00:16.800, the display position of the street dance is (1016,480) in the time period 00:00:16.800- - >00:00:17.400, and the display position of the street dance is (1000,496) in the time period 00:00:17.400- - >00:00: 18.000.

After the subtitle engine module 2110 analyzes the display positions of the subtitle content and the subtitle content at each time, the display positions of the subtitle content and the subtitle content at each time are sent to thesubtitle rendering module 2120, and thesubtitle rendering module 2120 renders the display positions of the received subtitle content and subtitle content at each time to obtain the subtitle displayed in the video, so that the street dance has a rotating function.

Fig. 4b is a schematic diagram of video pictures corresponding to different moments in time according to the present example.

As shown in fig. 4b, the subtitle content of "this is street dance" is displayed in a time period of 15-18 seconds after the start of the video, and fig. 4b also shows the display positions of the subtitle content displayed at different times in the time period.

FIG. 4b shows the display position of "this is street dance" at time 00:00:15.000, 00:00:15.400, 00:00:16.000, 00:00:16.400, 00:00:17.000, 00:00:18.000 of the video. At the time of 00:00:15.000 of the video, "this is street dance" at the subtitle display position of the video picture starts to be displayed. Then "this is the street dance" starts to rotate. At the time of 00:00:18.000 of the video, the display position of the "street dance" is rotated to the subtitle display position corresponding to the time of 00:00:15.000 of the video.

< example 2>

Based on the subtitle generating system provided by the embodiment of the present invention, a specific flow of subtitle generation will be described below by taking the example of forming a composite subtitle including an extended subtitle protocol content, and fig. 5a shows a flow diagram of a subtitle generating method of the example.

In this example, as shown in fig. 5a, on theserver 2100 side, the standard subtitle file generating module 2110 is configured to receive a video file and generate a corresponding standard subtitle file according to the video file. The standard subtitle file includes, for example, subtitle content of "00: 00:15.000- >00:00:18.000, which is street dance". The caption content is displayed at the caption display position during the 15 th to 18 th seconds after the start of the video, which is "street dance".

After the standard subtitle file generating module 2110 obtains the standard subtitle file, the standard subtitle file may be input to the compound subtitlefile generating module 2120.

The compound subtitlefile generating module 2120 is configured to receive the standard subtitle file provided by the standard subtitle file generating module 2110 and perform additional function processing on the standard subtitle file to obtain a compound subtitle file. In this example, the compound subtitle file includes text content and extended caption protocol content carried by a standard subtitle file. For example, the compound caption file includes 00:00:15.000- - >00:00:18.000 which is [ street dance ] { http:// xxx }. Wherein, the content of the extended caption protocol is { http:// xxx }. { http:// xxx } is the link address for jumping to other pages, and the display position of "street dance" indicates the region where the link is located. The link address and the area where the link is located constitute link information.

As shown in fig. 5a, on the side of the terminal device 2000, after receiving the compound subtitle file, the subtitle engine module 2110 parses the compound subtitle file to obtain subtitle content and extended subtitle protocol content. In this example, information for adding a jump link to the "street dance" is obtained based on the parsed content of the extended caption protocol, that is, the link address and the area where the link is located.

After the caption engine module 2110 parses the caption content and the extended caption protocol content, the caption engine module 2110 transmits the caption content and the extended caption protocol content to thecaption rendering module 2120, and thecaption rendering module 2120 renders the received caption content and the extended caption protocol content to obtain a caption displayed in the video.

After the caption engine module 2110 parses the caption content and the extended caption protocol content, the link information is also thrown to the interactive processing module 2130, so that the interactive processing module 2130 can sense an action of a user clicking an extended caption based on the link information and respond to the action to implement a corresponding link function. The link information may include, for example, a link address and an area where the link is located. Specifically, for the compound caption, when the user clicks on "street dance", the user can jump to the page corresponding to { http:// xxx }.

Fig. 5b is a schematic diagram of a video picture according to the present example.

According to fig. 5b, the subtitle content of "this is street dance" is displayed in the time period of 15-18 seconds after the video starts, and underlines are arranged below two characters of "street dance" to prompt the user to jump to the corresponding page when clicking the two characters of "street dance". Referring to fig. 5b, when the user clicks "street dance", the screen displayed by terminal device 2000 may jump to a page corresponding to { http:// xxx }.

< method examples >

Fig. 6 is a process flow diagram of a subtitle generating method according to one embodiment of the present invention. The method may be implemented by the server 1000 in fig. 1.

According to fig. 6, the method for generating subtitles may include the following steps S6100 to S6300.

Step S6100, obtain a standard subtitle file of the video, where the standard subtitle file includes subtitle content.

The standard subtitle files for video may be obtained directly from the video copyright side. Or, performing voice recognition processing on the video to obtain subtitle content, and then generating a standard subtitle file of the video by using the subtitle content. The subtitle content is text content obtained by identifying and processing audio data carried by a video. The standard caption file may be in any one of ass (advanced subtitle alpha), srt (subrip Text), webvtt (web Video Text tracks) formats.

Step S6200, the standard caption file is processed by the additional function to obtain a compound caption file, wherein the compound caption file comprises caption content and additional information content corresponding to the additional function.

The additional information content includes at least one of additional information content that realizes a subtitle change function and additional information content that realizes a subtitle link function.

In the embodiment of the present invention, the additional information content corresponding to the implementation of the subtitle change function is also referred to as meta subtitle protocol content, and the additional information content corresponding to the implementation of the additional linking function is also referred to as extended subtitle protocol content.

The subtitle change function includes at least one of a subtitle display position change function, a subtitle display color change function, and a subtitle display font change function.

The additional information content implementing the subtitle display position changing function may be a function implementing the subtitle display position changing function. The additional information content adopts a function to realize the function of changing the subtitle display position, so that the subtitle file sent to the terminal equipment is smaller, and the transmission efficiency of the subtitle file can be improved. For example, the function for implementing the subtitle display position changing function may be a [ cirlce (a, b, c, d) ] function. (a, b) represents a rotation center of the caption content, c represents a rotation radius of the caption content, and d represents a rotation duration of the caption content. Specifically, [ cirlcce (1000,480,16,3000) ] represents that the subtitle content makes a rotation of radius 16 around the center point (1000, 480) for a duration of 3000 milliseconds.

The subtitle link function comprises at least one of a function of realizing jumping to other pages through the subtitle link, a function of realizing the content corresponding to the attachment displayed on the current page through the subtitle link, and a function of adding a background layer to the subtitle through the subtitle link. In the above subtitle linking function, skipping means: the user clicks the link provided by the extended caption and can jump to another page; the accessories are as follows: the user clicks a link provided by the extended caption, and a small window or a dialog box is displayed on the current page to display the content corresponding to the attachment; the superposition means that: and clicking a link provided by the extended caption by the user, and adding a background layer for the extended caption, wherein the background layer can be a picture corresponding to the link.

When the subtitles are displayed, a user can realize the subtitle link function by clicking the subtitles, so that other functions of the subtitles except for display are added, interaction with the user is also increased, and user experience is improved.

And step S6300, sending the compound caption file to the terminal equipment for displaying the compound caption file.

In an embodiment of the present invention, the method for generating subtitles may further include the following steps S6400-S6500.

Step S6400, detecting whether an instruction for processing an additional function on the standard subtitle file exists or not to obtain a detection result;

and step S6500, under the condition that the detection result is negative, directly sending the standard subtitle file to the terminal equipment.

If the result of the detection is yes, the step S6200 is executed to perform the processing of adding the function to the standard subtitle file, so as to obtain the operation of the compound subtitle file.

Fig. 7 is a process flow diagram of a subtitle generating method according to one embodiment of the present invention. The method may be implemented by terminal device 2000 in fig. 1.

As shown in fig. 7, the subtitle generating method may include the following steps S7100 to S7300.

And step S7100, receiving the compound caption file sent by the server.

Step S7200, the compound caption file is analyzed to obtain the caption content and the additional information content corresponding to the additional function.

The subtitle content is text content obtained by identifying and processing audio data carried by a video.

The additional information content includes at least one of additional information content that realizes a subtitle change function and additional information content that realizes a subtitle link function. The content included in the subtitle change function and the content included in the subtitle link function may refer to an embodiment corresponding to a subtitle generating method implemented by a server, and are not described in detail herein.

Step S7300, rendering the subtitle content and the additional information content corresponding to the additional function to obtain a composite subtitle displayed in the video, so that the composite subtitle has the additional function.

In one embodiment, the additional information content includes link information implementing a subtitle link function. The link information may include, for example, a link address and an area where the link is located.

The subtitle generating method can also comprise the following steps of S7400-S7600.

And step S7400, interactive information for realizing human-computer interaction is formed according to the link information.

And step S7500, responding to the click operation of the composite caption, and detecting whether the click operation is performed according to the interactive information.

Step S7600, under the condition that whether the clicking operation is the operation according to the interactive information, providing the linked content according to the interactive information.

For example, the UI layer of the terminal device may receive the link information, sense an action of a user clicking a subtitle, and implement a corresponding link function in response to the action. Specifically, when the UI layer of the terminal device senses a click operation of a user, it is detected whether a click position corresponding to the click operation is located in an area where a link in the link information is located, and if so, the linked content is provided according to a link address in the link information.

Fig. 8 is a process flow diagram of a subtitle presentation method according to an embodiment of the present invention. The method may be implemented by terminal device 2000 in fig. 1.

According to fig. 8, the subtitle display method may include the following steps S8100 to S8300.

Step S8100, a compound caption file is obtained from a server, wherein the compound caption file comprises caption content and additional information content for realizing a caption change function.

The process of generating the compound subtitle file by the server may refer to the flow of the subtitle generating method shown in fig. 1, which is not described herein in detail.

Step S8200, reading the subtitle content in the compound subtitle file and the additional information content for realizing the subtitle change function.

In the present embodiment, the subtitle change function includes at least one of a subtitle display position change function, a subtitle display color change function, and a subtitle display font change function.

And S8300, generating and displaying a compound caption with a caption change function based on the caption content and the additional information content for realizing the caption change function, wherein the display effect of the compound caption comprises at least one of the change of the caption display position, the change of the caption display color and the change of the caption display font.

When the generated composite caption has a caption display position changing function, the caption content included in the composite caption is in a motion state when the composite caption is displayed.

When the generated composite subtitle has a subtitle display color change function, the color of subtitle content included in the composite subtitle may change when the composite subtitle is being displayed.

When the generated composite subtitle has a subtitle display font changing function, the font of the subtitle content included in the composite subtitle may be changed when the composite subtitle is displayed.

Fig. 9 is a process flow diagram of an interactive method of subtitles according to an embodiment of the present invention. The method may be implemented by terminal device 2000 in fig. 1.

As shown in fig. 9, the subtitle interaction method may include the following steps S9100-S9400.

Step S9100, a compound subtitle file is acquired from the server, wherein the compound subtitle file includes subtitle content and additional information content for implementing a subtitle linking function.

And step S9200, reading the subtitle content in the compound subtitle file and the additional information content for realizing the subtitle link function.

The additional information content includes link information implementing a subtitle link function. The link information may include, for example, a link address and an area where the link is located.

Step S9300 generates and displays a composite subtitle having a subtitle linking function based on the subtitle content and the additional information content for implementing the subtitle linking function.

In step S9400, a subtitle linking function is performed in response to an interactive instruction for a composite subtitle.

For example, when the terminal device receives an operation of clicking the subtitle content triggered by a user when the subtitle content included in the compound subtitle is being presented, a screen displayed by the terminal device may jump to the content linked to the additional information content.

< Caption Generation System >

The subtitle generating system of the present embodiment may include at least one server and at least one terminal device.

In an embodiment of the present invention, the memory of the server is configured to store instructions, where the instructions are configured to control the processor to execute the method for generating a subtitle according to the embodiment of the present invention. The memory of the terminal device is used for storing instructions used for controlling the processor to execute the generation method of the subtitles provided by the embodiment of the invention.

According to one embodiment of the invention, the multi-functionalization of the subtitles is realized, and only one showing function is provided.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A subtitle generating method implemented by a server comprises the following steps:

2. The method of claim 1, wherein the method further comprises:

3. The method of claim 1, wherein the obtaining the standard subtitle file for the video comprises:

4. The method of claim 1, wherein the additional function comprises at least one of a subtitle change function, a subtitle link function.

5. The method of claim 4, wherein the subtitle change function includes at least one of a subtitle display position change function, a subtitle display color change function, and a subtitle display font change function.

6. The method according to claim 4, wherein the subtitle link function comprises at least one of a function of jumping to other pages through the subtitle link, a function of displaying content corresponding to the attachment on the current page through the subtitle link, and a function of adding a background layer to the subtitle through the subtitle link.

7. A subtitle generating method is implemented by a terminal device and comprises the following steps:

receiving a composite caption file generated according to any one of the methods of claims 1-6, which is transmitted by a server;

8. The method of claim 7, wherein the additional function comprises at least one of implementing a subtitle change function, implementing a subtitle link function.

9. The method of claim 8, wherein the additional information content includes link information implementing a subtitle link function, wherein the method further comprises:

10. A subtitle display method comprises the following steps:

and generating and displaying a composite caption based on the caption content and the additional information content for realizing the caption change function, wherein the display effect of the composite caption has at least one of caption display position change, caption display color change and caption display font change.

11. A subtitle interaction method comprises the following steps:

12. A server comprising a memory and a processor, the memory for storing executable instructions; the processor is configured to perform the method according to any one of claims 1-6 under control of the instructions.

13. A terminal device comprising a memory and a processor, the memory for storing executable instructions; the processor is configured to perform the method according to any one of claims 7-9 under control of the instructions.

14. A subtitle generating system comprising the server of claim 12 and the terminal device of claim 13.