Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Referring to fig. 2, a schematic diagram of an implementation environment provided by an embodiment of the invention is shown. The implementation environment includes: a terminal 210 and a server 220.
The terminal 210 may be an electronic device such as a mobile phone, a tablet Computer, an e-book reader, a multimedia playing device, a wearable device, a Personal Computer (PC), and the like. Optionally, a client running audio playing software is installed in theterminal 210. The audio playing software has the functions of audio playing, message leaving and publishing, message leaving and displaying and the like. The audio playing software can be music playing software, listening book software, radio station software and the like.
The terminal 210 establishes a communication connection with the server 220 through a wired network or a wireless network.
The server 220 may be one server, a server cluster composed of a plurality of servers, or a cloud computing service center. Server 220 may be a backend server for providing backend servers for the clients described above.
In one example, as shown in fig. 3, when the server 220 is a server cluster composed of a plurality of servers, the server 220 includes: anaudio server 222, amessage server 224, and a user server 226.
Theaudio server 222 is configured to provide audio content information to the terminal, where the content information includes audio content and text content.
Themessage server 224 is configured to provide audio messages and message information to the terminal, where the message information includes at least one of the following: the identification of the message, the message time, the timestamp of the message and the user account corresponding to the message.
The user server 226 is used for providing user information to the terminal, the user information including at least one of: a Uniform Resource Locator (URL) for the user's avatar, and a user's nickname.
A communication connection may be established between themessage server 224 and the user server 226 via a network. Communication connections may also be established between theaudio server 222 and the terminal 210, and between themessage server 224 and the terminal 210 via a network. The network may be a wireless network or a wired network.
Referring to fig. 4A, a flowchart of an information displaying method according to an embodiment of the invention is shown. The method may be applied in the implementation environment shown in fig. 2. The method may include the steps of:
step 401, the terminal obtains the identifier of the target audio.
The audio may be songs, ghost stories, commentary, audiobooks, and the like. In the embodiment of the present invention, the description mainly takes the example that the audio is a song, but the invention is not limited to this. The target audio may be any audio, such as the audio to be played by the terminal. The identification of the audio is used for uniquely identifying the audio, and the identifications corresponding to different audios are different.
In one example, the terminal displays an audio list, and when a selection signal corresponding to a target audio in the audio list is acquired, an identification of the target audio is acquired. The selection signal refers to a signal triggered by an operation of selecting an audio performed by a user, for example, when the user selects a target audio through a click operation, the click operation triggers a selection signal corresponding to the target audio when the user clicks the target audio in the audio list, and the terminal acquires the selection signal. In another example, the terminal selects the next audio in the audio list as the target audio within a preset time period before the end of the currently played audio, and obtains the identifier of the target audio. The preset time period can be set according to actual requirements, such as within 10 seconds before the audio playing is finished.
In step 402, the terminal sends an information acquisition request to the server.
After acquiring the identifier of the target audio, the terminal sends an information acquisition request to the server, wherein the information acquisition request comprises the identifier of the target audio. Accordingly, the server receives the information acquisition request sent by the terminal.
And step 403, the server acquires the audio content and the message information of the target audio according to the identifier of the target audio.
The server stores the corresponding relation between the audio identifier and the audio content of the audio and the message information of the audio, and can inquire the corresponding relation according to the identifier of the target audio to acquire the audio content and the message information of the target audio. The audio content is an audio file in which audio sounds are recorded. The message information includes an identification and a timestamp of the message. The identification of the left message is used for uniquely identifying the left message, and the identifications corresponding to different left messages are different. The timestamp of the message is used to indicate the corresponding playing time of the message in the target audio, for example, if the timestamp of a message is 00:15.63, the playing time of the message corresponding to the target audio is 00: 15.63. Optionally, the playing time indicated by the time stamp is accurate to milliseconds. Optionally, the message information includes respective identifiers and timestamps of n messages, where n is a positive integer.
In the embodiment of the invention, the message is the speech of the user on the played audio in the audio playing process. The message may also be referred to as a barrage, comment, listening experience, etc.
Optionally, the message information of the target audio further includes at least one of: n messages, the message leaving time corresponding to the n messages and the user account corresponding to the n messages. The message means the message content. The message leaving time corresponding to the message is the time for issuing the message, for example, 14:28:01 in 7, 8 and 7 months in 2017. The user account corresponding to the left message is the user account which issues the left message.
Illustratively, taking the target audio as an example of a certain song, the left message information of the song is shown in the following table-1:
TABLE-1
As shown in table-1, the message information is indexed by the identifier of the message, and includes the correspondence between the identifier of the message, the user account, the identifier of the audio, the timestamp, the message, and the message time. Taking the first message in the table-1 as an example, the identifier of the first message is 001, the account number of the user who issues the message is 1234567, the message is issued while listening to the audio with the identifier 001, the corresponding playing time of the message in the audio is 00:18.48, the message content is "the lyrics are very profound", and the message issuing time is 2017, 7, 8, 14:28: 01. In addition, each message identification corresponds to a time stamp, but one time stamp can correspond to multiple message identifications, namely multiple messages can exist at one playing time of the target audio. The above-mentioned multiple messages may come from the same user or from different users. In addition, multiple messages from the same user may have different timestamps, i.e., the same user may post multiple messages at different play times.
Optionally, the server further obtains the text content of the target audio according to the identifier of the target audio. The audio content is an audio file recorded with audio sound, and the text content is text corresponding to the audio sound. For example, assuming that the target audio is a song, the audio content is an audio file of the song and the textual content is lyrics of the song. The text content of one audio comprises at least one text content, each text content corresponds to an initial playing time, and the initial playing time refers to the initial time for playing the text content in the audio.
Illustratively, taking target audio as a certain song as an example, the server obtains the identifier of the song sent by the terminal: after the audio 002, content information corresponding to the audio 002 is obtained, and the content information includes audio content and text content. The text content is shown in fig. 4B, and the text content shown in fig. 4B is lyrics of the song, where each lyric corresponds to a starting playing time, for example, the playing time corresponding to the lyric of "i love you" is 00: 18.48.
In step 404, the server sends an information acquisition response to the terminal.
And after acquiring the audio content and the message information of the target audio, the server sends an information acquisition response to the terminal, wherein the information acquisition response comprises the audio content and the message information of the target audio. Accordingly, the terminal receives the information acquisition response sent by the server.
Optionally, the information acquisition response further includes text content of the target audio.
Step 405, the terminal displays the message mark and the audio track corresponding to the audio content in the playing interface of the target audio.
And after receiving the information acquisition response, the terminal generates a corresponding audio track according to the audio content, wherein the audio track can reflect attributes such as tone, volume and the like of the audio content at each playing time. The message marks, the identifications of the messages and the timestamps of the messages are in one-to-one correspondence, each message mark corresponds to one message, and the display position of each message mark is determined according to the position of the timestamp of the message corresponding to the message mark in the audio track. In the embodiment of the invention, the message mark is used for marking a message, and the message is represented by using the message mark instead of directly displaying the message, so that the condition that all messages cannot be clearly displayed in a playing interface when the messages are too much is avoided. The message leaving mark may be a pattern such as a picture, a figure, or a character, which is not limited in the embodiment of the present invention.
Alternatively, the audio track corresponding to the audio content may only reflect the playing time of the audio content.
In a possible implementation manner, the terminal divides the playing time of the target audio into a plurality of time segments, and the time length of each time segment may be the same or different, for example, one audio with a playing time of two minutes, the terminal divides the playing time into 60 time segments, and the time length of each time segment is 2 seconds. And the terminal determines the message marks corresponding to the messages with the playing time corresponding to the time stamp in the same time period as a group of message marks. And displaying the audio track corresponding to the audio content in a playing interface, and displaying each group of message leaving marks according to the initial playing time of the time period to which each group of message leaving marks belongs, wherein the initial playing time of the time period to which each group of message leaving marks belongs is in one-to-one correspondence with the playing time in the audio track, and the message leaving marks in each group of message leaving marks are sequentially arranged and displayed. Optionally, the terminal divides a time period according to the text content of the target audio, and takes a time period between the initial playing time of each text content and the initial playing time of the next text content as a time period.
Optionally, a part or all of the text content of the target audio is also displayed in the playing interface of the target audio. Considering that more interface space is occupied for displaying all the text contents, a part of the text contents related to the current playing time, such as the text contents of the preset number before and after the current playing time, can be displayed in the playing interface of the target audio.
Optionally, step 405 is followed by step 406 as follows.
Step 406, the terminal displays the current playing time in the audio track and the message mark corresponding to the current playing time in a highlighted manner during the process of playing the audio content.
In the embodiment of the present invention, highlighting the current playing time in the audio track means displaying the current playing time in the audio track, for example, displaying the numerical value of the current playing time, displaying a part of the audio track before the current playing time and a part of the audio track after the current playing time in a distinguishing manner, and the like. Highlighting the message mark corresponding to the current playing time means displaying the message mark corresponding to the current playing time and other message marks differently, for example, adopting different colors, figures and styles, or adding marks on the message mark.
By highlighting the current playing time in the audio track and the message mark corresponding to the current playing time, the user can intuitively correspond the current playing time and the message mark corresponding to the current playing time.
Optionally, under the condition that the text content of the target audio is also displayed in the playing interface of the target audio, the terminal displays the current playing time in the audio track, the message mark corresponding to the current playing time, and the text content corresponding to the current playing time in a highlighted manner. Highlighting the text corresponding to the current playing time means displaying the text corresponding to the current playing time and other text differently, for example, using different colors and fonts, or underlining the text below. By highlighting the current playing time, the message mark corresponding to the current playing time and the text content corresponding to the current playing time in the audio track, the user can intuitively correspond the current playing time, the message mark corresponding to the current playing time and the text content corresponding to the current playing time.
In one example, the current playing time in the audio track, the message mark corresponding to the current playing time and the text content corresponding to the current playing time are highlighted in a time line manner. The time line refers to a line, such as a transverse straight line. Illustratively, please refer to fig. 4C, which shows a schematic interface diagram of the terminal playing audio in one embodiment. In the playinginterface 41, theaudio track 46 corresponding to the audio content is displayed on the left side of the playinginterface 41, and a part of theaudio track 46 before thecurrent playing time 43 is displayed differently from a part of the audio track after thecurrent playing time 43 in different colors. In the playinginterface 41, the message marks corresponding to the messages within the same time period at the playing time corresponding to the time stamp are arranged into a group of message marks, and each group of message marks is displayed in the middle of the playinginterface 41. The text content is displayed on the right part of theplayback interface 41. Thecurrent playing time 43 is marked on theaudio track 46 as 00:15.25, and thecurrent playing time 43 in theaudio track 46, themessage mark 44 corresponding to thecurrent playing time 43 and thetext content 45 corresponding to thecurrent playing time 43 are connected through thetime line 42, so that the purpose of highlighting is achieved. Further, thecharacter content 45 corresponding to thecurrent play time 43 is appropriately enlarged and displayed.
Optionally, the message information sent by the server to the terminal further includes: and the n messages respectively correspond to the user head portraits, or the URLs of the n messages respectively correspond to the user head portraits. The user avatar corresponding to a certain message is the avatar for publishing the user account corresponding to the message, and the URL of the user avatar corresponding to a certain message is the download address for publishing the avatar of the user account corresponding to the message. After the terminal acquires the user head portraits corresponding to the n messages, the user head portraits are used as message marks to be displayed; or after the terminal acquires the URLs of the user head portraits corresponding to the n messages, the user head portraits corresponding to the n messages are downloaded according to the URLs, and the user head portraits are displayed as message marks. The head portrait of the user is displayed as the message mark, so that the user can more intuitively know the user who leaves the message.
Optionally, when the server does not store the audio content and/or the text content of the target audio, the information acquisition response sent by the server to the terminal may include an audio address of the target audio, where the audio address is a download address for downloading the audio content and/or the text content of the target audio. And the terminal downloads and acquires the audio content and/or the text content of the target audio from other servers according to the received audio address.
In the embodiment of the invention, the time stamps of all messages are recorded, so that the audio track and the message marks corresponding to the audio content are displayed in the audio playing interface, each message mark corresponds to a message, and the display position of each message mark is determined according to the position of the time stamp corresponding to the message mark in the audio track, so that the relevance between the messages and the playing time is embodied, and the messages can be directed to the audio content at a certain playing time instead of the whole audio. In addition, the messages are displayed in a message marking mode, so that the interface space can be saved, and the optimization of the interface layout is facilitated.
In addition, the current playing time in the audio track and the message mark corresponding to the current playing time are highlighted in the process of playing the audio content, so that the relevance between the current playing time and the message is more intuitively reflected.
In addition, in the process of playing the audio content, the current playing time in the audio track, the message mark corresponding to the current playing time and the text content corresponding to the current playing time are highlighted, so that the relevance among the current playing time, the message and the text content is more intuitively reflected.
It should be noted that, in the embodiment of fig. 4A, only the audio content, the text content, and the message information of the target audio requested by the terminal from the server are taken as an example. In other embodiments, the terminal may also store the audio content, the text content, and the message information of the target audio in advance in a local, obtain the audio content, the text content, and the message information of the target audio locally, and perform subsequent display, which is not limited in this embodiment of the present invention.
In an alternative embodiment provided based on the embodiment of fig. 4A, as shown in fig. 5, in this embodiment, a message updating process is mainly described, which includes the following steps:
step 501, in the process of playing the audio content, the terminal sends a message updating request to the server at preset time intervals.
The message updating request comprises the identification of the target audio, and is used for requesting to acquire updated message information.
Optionally, the message update request further includes: and the latest message leaving time refers to the message leaving time which is closest to the current time in the message leaving information received by the terminal.
The preset time interval can be preset according to actual requirements. Illustratively, the preset time interval is 10 seconds, and two adjacent preset time intervals may be the same or different.
Step 502, the server obtains updated message information of the target audio.
The updated message information includes an identification and a timestamp of the updated message. Optionally, the updated message information includes respective identifiers and timestamps of m updated messages, where m is a positive integer. The updated message identification refers to the identification of the message of the target audio which is not sent to the terminal before. After receiving a message updating request sent by the terminal, the server determines the identifier of the message corresponding to the target audio sent to the terminal according to the identifier of the target audio, and then obtains the identifier of the updated message according to the identifier of the sent message.
Optionally, in a case that the message update request further includes the latest message leaving time, the server may obtain, according to the latest message leaving time, an identifier of a message leaving the target audio after the latest message leaving time, as the identifier of the updated message. By the method, the complexity of the server for acquiring the updated message identification can be simplified, the server side does not need to store the message identification of each audio sent to each terminal, and a large amount of storage space is saved.
Optionally, the updated message further includes at least one of the following: m messages, the message leaving time corresponding to the m messages and the user account corresponding to the m messages. Optionally, the updated message information further includes: and (3) the user head portraits corresponding to the m messages, or the URLs of the user head portraits corresponding to the m messages.
Step 503, the server sends a message update response to the terminal.
The message updating response comprises the updated message information. Correspondingly, the terminal receives the message updating response sent by the server.
And step 504, adding a message display mark in the playing interface by the terminal.
The added and displayed message marks, the updated message marks and the updated timestamp of the messages correspond to one another, and each message mark corresponds to one message. And after receiving the message updating response, the terminal adds the message mark corresponding to the updated message into the playing interface according to the timestamp of the message in the received updated message information.
In the embodiment of the invention, the message is updated in real time in the audio playing process, so that the timeliness of the displayed message is ensured.
In an alternative embodiment provided based on the embodiment of fig. 4A, as shown in fig. 6A, in this embodiment, a description is mainly given of a message obtaining process, which includes the following steps:
step 601, if the terminal acquires the trigger signal corresponding to the first message mark, sending a message acquisition request to the server.
After clicking the first message mark, the terminal acquires a trigger signal corresponding to the first message mark, and then sends a message acquisition request to the server, wherein the message acquisition request comprises: and the identifier of the target audio and the identifier of the first message corresponding to the first message mark. The first message leaving mark may be any one of all message leaving marks displayed in the playing interface.
Step 602, the server obtains the first message according to the identifier of the first message.
Because each message corresponds to an identifier of the message, all the servers can acquire the first message according to the identifier of the first message.
Optionally, the server may further obtain, according to the identifier of the first message, a user account corresponding to the first message or a user nickname corresponding to the first message.
Step 603, the server sends a message acquisition response to the terminal.
The message obtaining response comprises a first message. Optionally, the message obtaining response further includes a user account corresponding to the first message or a user nickname corresponding to the first message. Correspondingly, the terminal receives the message acquisition response sent by the server.
And step 604, the terminal displays a message display interface on the upper layer of the playing interface and displays the first message in the message display interface.
Optionally, the terminal may further display a user account corresponding to the first message or a user nickname corresponding to the first message in the message display interface.
Optionally, the message obtaining request sent by the terminal to the server includes: and the identification of the target audio and the identifications of the messages corresponding to all the message marks in the time period to which the first message mark belongs. And the server acquires the messages corresponding to all the message marks in the time period to which the first message mark belongs according to the identifier of the message, and sends the acquired messages to the terminal in a message acquisition response. And the terminal displays the messages corresponding to all the message marks in the time period to which the received first message mark belongs in a message display interface.
Optionally, if the message information received by the terminal includes the first message, the terminal does not send a message obtaining request after obtaining the trigger signal corresponding to the first message mark, and directly displays the first message in the message display interface.
Exemplarily, referring to fig. 6B, which shows a schematic diagram of amessage display interface 47 in an embodiment, after acquiring a trigger signal corresponding to amessage mark 44, a terminal displays all messages at a play time corresponding to a first message in themessage display interface 47, where as shown in fig. 6B, a first message is amessage 49 corresponding to auser nickname 48 and a bean: the music is too pleasant.
In this embodiment, the presentation of the message content is realized.
In an alternative embodiment provided based on the embodiment of fig. 4A, as shown in fig. 7, in this embodiment, a message publishing process is mainly described, which includes the following steps:
and 701, in the process of playing the audio content, if a trigger signal corresponding to the message publishing control is obtained, displaying a message publishing interface.
When a user clicks the message publishing control in the playing interface, the terminal acquires a trigger signal corresponding to the message publishing control and then displays the message publishing interface. In the message publishing interface, the user inputs a second message to be published.
Optionally, in the message publishing interface, the terminal displays the name of the target audio, the performer, the playing time corresponding to the message to be published, and the text corresponding to the playing time.
Referring to fig. 7B, which is a schematic diagram of amessage posting interface 71 in an embodiment, as shown in fig. 7B, in themessage posting interface 71, a terminal displays a song name 72: save Me, performer 73: zhang three, play time 74: 00:15.25, lyrics 75: you are shining and the user can enter a message to be published via thekeypad 76, for example, thesecond message 77 is "really good at".
And step 702, the terminal acquires a second message input in the message publishing interface.
And after the user inputs a second message to be published, the terminal acquires the second message.
Optionally, after the terminal obtains the second message, it detects whether there is a Cross-Scripting attack (XSS) in the obtained second message, where XSS is a security vulnerability attack of a website application program, and generally refers to a vulnerability left during web page development, and injects a malicious instruction code into a web page by a smart method, so that a user loads and executes a web page program maliciously manufactured by an attacker. An attacker can entrain malicious HyperText Markup Language (HTML) script code in input content through normal input means. When the page content of the malicious script is injected on the access server of the victim, the malicious script is executed. Therefore, it is necessary to detect whether XSS exists in the message content. If XSS exists, performing escape processing on the special character in the second statement, illustratively, performing escape processing on the special character < > & "" when XSS exists in the second statement; if not, no processing is performed.
Step 703, the terminal sends a message publishing request to the server.
The message publishing request comprises the identification of the target audio, the second leave message and the timestamp of the second message. The message publishing request is used for indicating the server to store the second message.
Optionally, the message publishing request further includes at least one of the following: the message leaving time corresponding to the second message and the user account corresponding to the second message.
Step 704, the server generates an identifier of the second message according to the identifier of the target audio, the second message and the timestamp of the second message.
The server can determine a message according to the identification of the audio, the message and the timestamp of the message, and therefore the identification of the second message is generated according to the identification of the target audio, the second message and the timestamp of the second message.
Optionally, the server can also generate an identifier of the second message according to the identifier of the target audio, the message leaving time corresponding to the second message, the second leave word of the user account corresponding to the second message, and the timestamp of the second message.
Optionally, after the server obtains the second message, detecting whether the second message has XSS, and if yes, performing escape processing on the special character in the second message; if not, no processing is performed.
Step 705, the server stores the identifier of the target audio, the second message, the timestamp of the second message, and the identifier of the second message.
And after the identification corresponding to the second message is determined, the server stores the identification of the target audio, the second message, the timestamp of the second message and the identification of the second message.
Optionally, the server can further store the identifier, the timestamp, the message leaving time, and the user account corresponding to the second message.
Step 706, the server sends a message issue response to the terminal.
The message publishing response comprises an identifier corresponding to the second message.
And step 707, the terminal adds the message mark corresponding to the second message to the playing interface.
Each message mark corresponds to a message. And after receiving the message issuing response, the terminal adds the message mark corresponding to the second message into the playing interface according to the timestamp of the second message, and records the corresponding relation between the identifier of the second message and the message mark corresponding to the second message.
In this embodiment, the message content is published. In addition, the mark of the target audio, the second leave word and the time stamp of the second leave word are carried in the leave word publishing request, so that the playing time, the second leave word and the text content can be displayed in a correlated manner when the second leave word is displayed subsequently.
Please refer to fig. 8A/8B/8C/8D, which shows a flowchart of an information displaying method according to another embodiment of the present invention. The method may be applied in the implementation environment shown in fig. 3.
The following introduces the initial playing process, the message updating process, the message acquiring process and the message publishing process in the embodiments, respectively.
1. As shown in fig. 8A, the initial play process includes the following steps:
step 801, the terminal acquires the identifier of the target audio.
Step 802, the terminal sends a content information acquisition request to the audio server.
The content information acquisition request includes an identification of the target audio.
In step 803, the terminal sends a message information acquisition request to the message server.
The message information acquisition request comprises the identification of the target audio.
Step 803 can be executed after step 802, or before step 802, or simultaneously with step 802, which is not limited in this embodiment of the present invention.
And step 804, the audio server acquires the content information of the target audio according to the identifier of the target audio.
The content information of the target audio comprises the audio content and the text content of the target audio.
Step 805, the message server obtains the message information of the target audio according to the identifier of the target audio.
The message information comprises respective identifications and time stamps of the n messages.
Step 806, the message server sends a user information obtaining request to the user server.
The user information acquisition request comprises the user accounts of the n messages.
In step 807, the user server obtains the user avatar URL or user avatar corresponding to the user accounts with n messages.
And the user server acquires the URLs of the user head portraits or the user head portraits corresponding to the n user accounts with the messages according to the n user accounts with the messages.
Step 808, the user server sends a user information acquisition response to the message server.
The user information acquisition response comprises the URLs or user head portraits of the user corresponding to the n left-word user accounts.
In step 809, the audio server sends a content information acquisition response to the terminal.
The content information acquisition response includes the content information of the target audio.
Step 810, the message server sends a message information acquisition response to the terminal.
The message information acquisition response comprises message information of the target audio.
The message information further includes at least one of: n messages, the message leaving time corresponding to the n messages and the user account corresponding to the n messages.
Optionally, the message information further includes: and the n messages respectively correspond to the user head portraits, or the URLs of the n messages respectively correspond to the user head portraits.
Step 811, the terminal displays the audio track, the n message marks and the text content corresponding to the audio content in the playing interface of the target audio.
Alternatively, when the audio server does not store the content information of the target audio, the audio address, which is a download address for downloading the content information of the target audio, may be included in the content information acquisition response acquired by the audio server and transmitted to the terminal. And the terminal acquires the content information of the target audio from other servers according to the received audio address.
2. As shown in fig. 8B, the message update process includes the following steps:
in step 812, the terminal sends a message update request to the message server at preset time intervals during the process of playing the audio content.
In step 813, the message server obtains updated message information of the target audio.
The updated message information comprises respective identification and time stamp of m updated messages, and m is a positive integer.
In step 814, the message server sends a user update request to the user server.
The user updating request comprises the user accounts of the m messages.
Step 815, the user server obtains the user head portrait URL or the user head portrait corresponding to the user account of the m messages.
And the user server acquires user head portraits URL or user head portraits corresponding to the user accounts with the m messages according to the user accounts with the m messages.
Step 816, the user server sends a user update response to the message server.
The user updating response comprises user head portraits URL or user head portraits corresponding to the user accounts of the m messages.
Step 817, the message server sends a message update response to the terminal.
The message information acquisition response includes the updated message information.
The message information further includes at least one of: the m messages, the message leaving time corresponding to the m messages and the user account corresponding to the m messages.
Optionally, the message information further includes: and (3) the user head portraits corresponding to the m messages, or the URLs of the user head portraits corresponding to the m messages.
Step 818, the terminal adds and displays m message leaving marks in the playing interface.
3. As shown in fig. 8C, the message obtaining process includes the following steps:
step 819, if the terminal obtains the trigger signal corresponding to the first message flag, then a message obtaining request is sent to the message server.
Step 820, the message server obtains the first message according to the identifier of the first message.
In step 821, the message server sends a message obtaining response to the terminal.
Step 822, the terminal displays a message display interface on the upper layer of the playing interface and displays the first message in the message display interface.
4. As shown in fig. 8D, the message publishing process includes the following steps:
step 823, in the process of playing the audio content, if the terminal acquires the trigger signal corresponding to the message publishing control, displaying a message publishing interface.
Step 824, the terminal obtains the second message input in the message publishing interface.
Step 825, the terminal sends a message publishing request to the message server.
And step 826, the message server generates an identifier of the second message according to the identifier of the target audio, the second leave word and the timestamp of the second message.
In step 827, the message server stores the identifier of the target audio, the second message, the timestamp of the second message, and the identifier of the second message.
Step 828, the message server sends a message publishing response to the terminal.
Step 829, the terminal adds the message mark corresponding to the second message to the playing interface.
The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.
Referring to fig. 9, a block diagram of an information display device according to an embodiment of the invention is shown. The device has the function of implementing the terminal side method in the above embodiments, and the function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The apparatus may include: aninformation acquisition module 901 and aninformation display module 902.
Theinformation obtaining module 901 is configured to obtain audio content of a target audio and message information, where the message information includes an identifier and a timestamp of a message, and the timestamp of the message is used to indicate a corresponding playing time of the message in the target audio.
Aninformation display module 902, configured to display a message mark and a track corresponding to the audio content in a playing interface of the target audio, where the message mark, the identifier of the message, and the timestamp of the message are in one-to-one correspondence, and a display position of each message mark is determined according to a position of the timestamp of the message corresponding to the message mark in the track.
In the embodiment of the invention, the time stamps of all messages are recorded, so that the audio track and the message marks corresponding to the audio content are displayed in the audio playing interface, each message mark corresponds to a message, and the display position of each message mark is determined according to the position of the time stamp corresponding to the message mark in the audio track, so that the relevance between the messages and the playing time is embodied, and the messages can be directed to the audio content at a certain playing time instead of the whole audio. In addition, the messages are displayed in a message marking mode, so that the interface space can be saved, and the optimization of the interface layout is facilitated.
Optionally, theinformation displaying module 902 is further configured to: and in the process of playing the audio content, highlighting the current playing time in the audio track and the message mark corresponding to the current playing time.
Optionally, theinformation obtaining module 901 is further configured to: and acquiring the text content of the target audio. Aninformation display module 902, further configured to: and in the process of playing the audio content, highlighting the current playing time in the audio track, the message mark corresponding to the current playing time and the text content corresponding to the current playing time.
Optionally, theinformation obtaining module 901 is further configured to: acquiring an identifier of the target audio; sending an information acquisition request to a server, wherein the information acquisition request comprises the identifier of the target audio; and receiving an information acquisition response sent by the server, wherein the information acquisition response comprises the audio content of the target audio and the message information.
Optionally, theinformation obtaining module 901 is further configured to: in the process of playing the audio content, sending a message updating request to a server at preset time intervals, wherein the message updating request comprises the identifier of the target audio; and receiving a message updating response sent by the server, wherein the message updating response comprises updated message information of the target audio, and the updated message information comprises an updated message identifier and a timestamp. Aninformation display module 902, further configured to: and adding a message display mark in the playing interface, wherein the message display mark, the identification of the updated message and the timestamp of the updated message are in one-to-one correspondence.
Optionally, theinformation obtaining module 901 is further configured to: when a trigger signal corresponding to the first message mark is acquired, a message acquisition request is sent to a server, wherein the message acquisition request comprises: the identification of the target audio and the identification of the first message corresponding to the first message mark; and receiving a message obtaining response sent by the server, wherein the message obtaining response comprises the first message. Aninformation display module 902, further configured to: and displaying a message display interface on the upper layer of the playing interface, and displaying the first message in the message display interface.
Optionally, theinformation displaying module 902 is further configured to: and in the process of playing the audio content, when a trigger signal corresponding to the message publishing control is acquired, displaying a message publishing interface. Theinformation obtaining module 901 is further configured to: acquiring a second message input in the message publishing interface; and sending a message publishing request to a server, wherein the message publishing request comprises the identifier of the target audio, the second message and the timestamp of the second message.
Referring to fig. 10, a block diagram of an information display device according to another embodiment of the invention is shown. The device has the function of implementing the server-side method in the above embodiment, and the function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The apparatus may include: arequest receiving module 1001, aninformation obtaining module 1002 and aresponse sending module 1003.
Arequest receiving module 1001, configured to receive an information obtaining request sent by a terminal, where the information obtaining request includes an identifier of a target audio.
Theinformation obtaining module 1002 is configured to obtain the audio content of the target audio and the message information according to the identifier of the target audio, where the message information includes an identifier of a message and a timestamp, and the timestamp of the message is used to indicate a corresponding playing time of the message in the target audio.
Aresponse sending module 1003, configured to send an information obtaining response to the terminal, where the information obtaining response includes the audio content of the target audio and the message information, so that the terminal displays a message mark and a track corresponding to the audio content in a playing interface of the target audio, where the message mark, an identifier of the message, and a timestamp of the message are in one-to-one correspondence, and a display position of each message mark is determined according to a position of the timestamp of the message corresponding to the message mark in the track.
In the embodiment of the invention, the time stamps of all messages are recorded, so that the audio track and the message marks corresponding to the audio content are displayed in the audio playing interface, each message mark corresponds to a message, and the display position of each message mark is determined according to the position of the time stamp corresponding to the message mark in the audio track, so that the relevance between the messages and the playing time is embodied, and the messages can be directed to the audio content at a certain playing time instead of the whole audio. In addition, the messages are displayed in a message marking mode, so that the interface space can be saved, and the optimization of the interface layout is facilitated.
Optionally, therequest receiving module 1001 is further configured to receive a message update request sent by the terminal, where the message update request includes an identifier of the target audio. Theinformation obtaining module 1002 is further configured to obtain updated message information of the target audio according to the identifier of the target audio, where the updated message information includes an identifier and a timestamp of an updated message. Theresponse sending module 1003 is further configured to send a message update response to the terminal, where the message update response includes the updated message information, so that the terminal adds and displays a message mark in the playing interface, and the message mark added and displayed, the identifier of the updated message, and the timestamp of the updated message are in one-to-one correspondence.
Optionally, therequest receiving module 1001 is further configured to receive a message obtaining request sent by the terminal, where the message obtaining request includes: and the identifier of the target audio and the identifier of the first message corresponding to the first message mark. Theinformation obtaining module 1002 is further configured to obtain the first message according to the identifier of the first message. Theresponse sending module 1003 is further configured to send a message obtaining response to the terminal, where the message obtaining response includes the first message, so that the terminal displays a message display interface on the upper layer of the play interface and displays the first message in the message display interface.
Optionally, therequest receiving module 1001 is further configured to receive a message publishing request sent by the terminal, where the message publishing request includes the identifier of the target audio, the second message, and a timestamp of the second message. Theinformation obtaining module 1002 is further configured to generate an identifier of a second message according to the identifier of the target audio, the second message, and the timestamp of the second message. The device further comprises: and an information storage module. And the information storage module is used for correspondingly storing the identifier of the target audio, the second message, the timestamp of the second message and the identifier of the second message.
An exemplary embodiment of the present invention also provides an information display system, including: a terminal and a server. The terminal includes an information display device provided as an embodiment shown in fig. 9. The server includes an information display device provided as the embodiment shown in fig. 10.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Referring to fig. 11, a schematic structural diagram of a terminal according to an embodiment of the present invention is shown. The terminal is used for implementing the information display method on the terminal side provided by the embodiment. Specifically, the method comprises the following steps:
terminal 1100 can include RF (Radio Frequency)circuitry 1110,memory 1120 including one or more computer-readable storage media,input unit 1130,display unit 1140,sensors 1150,audio circuitry 1160, WiFi (wireless fidelity)module 1170,processor 1180 including one or more processing cores, andpower supply 1190. Those skilled in the art will appreciate that the terminal structure shown in fig. 11 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
RF circuit 1110 may be used for receiving and transmitting signals during a message transmission or communication process, and in particular, for receiving downlink messages from a base station and then processing the received downlink messages by one ormore processors 1180; in addition, data relating to uplink is transmitted to the base station. In general,RF circuitry 1110 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, and the like. In addition, theRF circuitry 1110 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), and the like.
Thememory 1120 may be used to store software programs and modules, and theprocessor 1180 may execute various functional applications and data processing by operating the software programs and modules stored in thememory 1120. Thememory 1120 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the terminal 1100, and the like. Further, thememory 1120 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, thememory 1120 may also include a memory controller to provide theprocessor 1180 and theinput unit 1130 access to thememory 1120.
Theinput unit 1130 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, theinput unit 1130 may include animage input device 1131 andother input devices 1132. Theimage input device 1131 may be a camera or a photoelectric scanning device. Theinput unit 1130 may includeother input devices 1132 in addition to theimage input device 1131. In particular,other input devices 1132 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
Thedisplay unit 1140 may be used to display information input by or provided to the user and various graphical user interfaces of the terminal 1100, which may be made up of graphics, text, icons, video, and any combination thereof. TheDisplay unit 1140 may include aDisplay panel 1141, and optionally, theDisplay panel 1141 may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
The terminal 1100 can also include at least onesensor 1150, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that adjusts the brightness of thedisplay panel 1141 according to the brightness of ambient light, and a proximity sensor that turns off thedisplay panel 1141 and/or the backlight when the terminal 1100 moves to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured in the terminal 1100, detailed descriptions thereof are omitted.
Audio circuitry 1160,speakers 1161, andmicrophone 1162 may provide an audio interface between a user and terminal 1100. Theaudio circuit 1160 may transmit the electrical signal converted from the received audio data to thespeaker 1161, and convert the electrical signal into a sound signal for output by thespeaker 1161; on the other hand, themicrophone 1162 converts the collected sound signal into an electric signal, receives it by theaudio circuit 1160, converts it into audio data, processes it by the audiodata output processor 1180, and transmits it to, for example, another terminal via theRF circuit 1110, or outputs it to thememory 1120 for further processing.Audio circuitry 1160 may also include an earbud jack to provide peripheral headset communication with terminal 1100.
WiFi belongs to short-distance wireless transmission technology, and the terminal 1100 can help the user send and receive e-mails, browse web pages, access streaming media, etc. through theWiFi module 1170, and it provides the user with wireless broadband internet access. Although fig. 11 shows theWiFi module 1170, it is understood that it does not belong to the essential constitution of the terminal 1100, and can be omitted entirely as needed within the scope not changing the essence of the invention.
Theprocessor 1180 is a control center of the terminal 1100, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the terminal 1100 and processes data by operating or executing software programs and/or modules stored in thememory 1120 and calling data stored in thememory 1120, thereby performing overall monitoring of the mobile phone. Optionally,processor 1180 may include one or more processing cores; preferably, theprocessor 1180 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated withinprocessor 1180.
Terminal 1100 can also include a power supply 1190 (e.g., a battery) for providing power to various components, which can be logically coupled toprocessor 1180 via a power management system that can be configured to manage charging, discharging, and power consumption.Power supply 1190 may also include one or more dc or ac power supplies, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, or any other component.
Although not shown, the terminal 1100 may further include a bluetooth module or the like, which is not described in detail herein.
Specifically, in this embodiment, the terminal 1100 further includes a memory, and at least one instruction, at least one program, a code set, or an instruction set, where the at least one instruction, the at least one program, the code set, or the instruction set is stored in the memory and configured to be executed by one or more processors, so as to implement the functions of the steps at the terminal side in the information display method.
Referring to fig. 12, a schematic structural diagram of a server according to an embodiment of the present invention is shown. The server is used for implementing the server-side information display method provided by the above embodiment. Specifically, the method comprises the following steps:
theserver 1200 includes a Central Processing Unit (CPU)1201, asystem memory 1204 including a Random Access Memory (RAM)1202 and a Read Only Memory (ROM)1203, and asystem bus 1205 connecting thesystem memory 1204 and thecentral processing unit 1201. Theserver 1200 also includes a basic input/output system (I/O system) 1206 to facilitate transfer of information between devices within the computer, and amass storage device 1207 for storing anoperating system 1213,application programs 1214, and other program modules 1215.
The basic input/output system 1206 includes adisplay 1208 for displaying information and aninput device 1209, such as a mouse, keyboard, etc., for a user to input information. Wherein thedisplay 1208 andinput device 1209 are connected to thecentral processing unit 1201 through an input-output controller 1210 coupled to thesystem bus 1205. The basic input/output system 1206 may also include an input/output controller 1210 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 1210 also provides output to a display screen, a printer, or other type of output device.
Themass storage device 1207 is connected to thecentral processing unit 1201 through a mass storage controller (not shown) connected to thesystem bus 1205. Themass storage device 1207 and its associated computer-readable media provide non-volatile storage for theserver 1200. That is, themass storage device 1207 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. Thesystem memory 1204 andmass storage device 1207 described above may be collectively referred to as memory.
Theserver 1200 may also operate as a remote computer connected to a network via a network, such as the internet, in accordance with various embodiments of the present invention. That is, theserver 1200 may be connected to thenetwork 1212 through anetwork interface unit 1211 coupled to thesystem bus 1205, or thenetwork interface unit 1211 may be used to connect to other types of networks or remote computer systems.
The memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions configured to be executed by one or more processors to implement the functions of the various steps at the server side in the above-described information display method.
In an exemplary embodiment, there is also provided a computer-readable storage medium having stored therein at least one instruction, at least one program, a code set, or a set of instructions which, when executed by a processor of a terminal, implement the functions of the respective steps on the terminal side in the above-described information display method.
In an exemplary embodiment, there is also provided a computer-readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions which, when executed by a processor of a server, implement the functions of the respective steps at the server side in the above-described information display method.
Alternatively, the computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The above description is only exemplary of the present invention and should not be taken as limiting the invention, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.