CN101261864B - A method and system for mixing recording voice at a mobile terminal - Google Patents

Abstract

The invention discloses a method and a system for realizing the synthesis of sound recording on a mobile terminal. The method includes the following steps: select a multimedia document to be dubbed on a terminal; carry out the audio and video splitting of the multimedia document; decode the split audio data and video data to generate the decoded video data and the audio data in linear pulse code modulation recording format and back up the decoded audio data; the video data before the decoding can be directly used as the video backup if the video formats before and after the recoding synthesization are the same, otherwise, the decoded video data are encoded and then taken as the backup; record on the terminal to generate the audio data in the linear pulse code modulation recording format, the recorded audio data and the decoded audio data in the linear pulse code modulation recording format are synthesized to be encoded, then the video data after backup and the audio data after synthesized encoding are merged and encoded and output together. The method and the system cause the customer to realize the synthesizing function of sound recording by the mobile terminal.

Description

A kind of synthetic method and system of recording of on portable terminal, realizing

Technical field

The present invention relates to portable terminal recording synthetic technology, more particularly, relate to a kind of admission voice or music of being used on portable terminal and add in original music, animation or the video and go, in order to realize dubbing or the method and system of the function of dubbing in background music.

Background technology

In current mobile terminal (mobile phone), the multimedia function that can realize is a lot, can playing back music, animation and video record etc.; Portable terminal also can be realized functions such as recording simultaneously.If recording can be synthesized in the existing multimedia file, dub as giving one section animation, join perhaps for one section music and sing, then can be used for making the individual character the tinkle of bells or be used as multimedia message sending to kith and kin, will provide novel more interesting experience for mobile terminal user.

Summary of the invention

Technical matters solved by the invention is to provide a kind of method and system that recording is synthetic of realizing on portable terminal, to realize that the recording complex functionality that provides on the portable terminal is provided the user, the problem that can oneself record and dub for existing multimedia material.

In order to realize the problems referred to above, the invention provides a kind of method that recording is synthetic that on portable terminal, realizes, be applied to support it is characterized in that in the portable terminal of multi-medium data, may further comprise the steps:

Select to be its multimedia file of dubbing on the described portable terminal, described multimedia file is carried out the fractionation of Voice ﹠ Video, the voice data and the video data that split the back generation are decoded, generate the voice data of decoded video data and linear pulse-code modulation recording form, and this decoded voice data exported and back up, if the video format before and after recording is synthetic is identical, then the data before the video decode are directly backed up as video, otherwise, carry out again being used as backup again after the respective coding with the video data behind the video decode;

On described portable terminal, begin recording, generate the voice data of linear pulse-code modulation recording form, the voice data of this voice data recorded and described decoded linear pulse-code modulation recording form is synthesized together and encodes, voice data behind video data after the described backup and the described composite coding is merged coding, and output.

Method of the present invention, wherein, the voice data of described voice data that this is recorded and described decoded linear pulse-code modulation recording form is synthesized together, further be the audio frequency superposition algorithm of utilization pulse-code modulation recording form, the voice data of this voice data recorded and described decoded linear pulse-code modulation recording form is synthesized together.

Further, described multimedia file comprises music, animation and the video clip of the various forms of supporting on the described portable terminal.

Before the step of described method, also comprise judging whether insert earphone on the described portable terminal, then point out the user to insert earphone and regularly, then do not finish the step of this method and withdraw from as not inserting earphone in the stipulated time as not inserting earphone; Go into earphone if any inserting earphone or stipulated time interpolation, step below then continuing to carry out.

In order to realize the problems referred to above, the present invention also provides a kind of system that recording is synthetic that realizes on portable terminal, be applied to support in the portable terminal of multi-medium data, it is characterized in that, comprise multi-media decoding and encoding module, audio coding decoding module, coding and decoding video module, audio frequency backup module, video backup module, audio frequency load module, audio frequency output module, video output module, audio frequency laminating module, wherein

Described multi-media decoding and encoding module is used for will splitting and merging for the voice data of its multimedia file of dubbing and video data selecting on the described portable terminal;

Described audio coding decoding module is used for the voice data that described multi-media decoding and encoding module fractionation back generates is decoded, and the voice data that export described audio frequency laminating module stack back is encoded;

Described coding and decoding video module, be used for the video data that described multi-media decoding and encoding module fractionation back generates is decoded, if the video format before and after recording is synthetic is identical, then the data before the video decode are directly issued described video backup module, otherwise, carry out again issuing described video backup module again after the respective coding with the video data behind the video decode;

Described coding and decoding video module, be used for the video data that described multi-media decoding and encoding module fractionation back generates is decoded, if the video format before and after recording is synthetic is identical, then the data before the video decode are directly issued described video backup module, otherwise, carry out again issuing described video backup module again after the respective coding with the video data behind the video decode;

Described audio frequency backup module is used for the voice data of described audio coding decoding module output is backed up;

Described video backup module is used for described multi-media decoding and encoding module is backed up the video data of multi-medium data fractionation back output or the video data behind the described coding and decoding video module coding;

Described audio frequency load module is used for receiving the recording data, generates the voice data of linear pulse-code modulation recording form;

Described audio frequency laminating module is used for the voice data of described audio frequency load module generation and the voice data of the decoded linear pulse-code modulation recording form of described audio coding decoding module are synthesized.

System of the present invention, wherein, described audio frequency laminating module, be further used for using the audio frequency superposition algorithm of pulse-code modulation recording form, the voice data of described audio frequency load module generation and the voice data of the decoded linear pulse-code modulation recording form of described audio coding decoding module are synthesized.

Described system further comprises:

The audio frequency output module is used to export the decoded voice data of described audio coding decoding module;

The video output module is used to export the decoded video data of described coding and decoding video module.

Further, described multimedia file comprises music, animation and the video clip of the various forms of supporting on the described portable terminal.

Further comprise a judge module and timer, wherein, described judge module, be used to judge and whether insert earphone on the described portable terminal, as do not insert earphone then point out the user to insert earphone and start as described in timer regularly, do not insert earphone as described in the stipulated time of timer and then finish to log off; Go into earphone if any the stipulated time interpolation of inserting earphone or described timer, then indicate the work of described multi-media decoding and encoding module.

Compared with prior art, the present invention can add the audio that the user needs in original multimedia material to, so the no longer only simple multimedia files such as playing back music of portable terminal (mobile phone) can also be used to recording synthetic multimedia, produce personalized the tinkle of bells and multimedia short-movie.

Description of drawings

Fig. 1 is the method flow diagram that recording is synthetic of realizing on portable terminal of the present invention;

Fig. 2 is the system construction drawing that recording is synthetic of realizing on portable terminal of the present invention.

Embodiment

The present invention provides a kind of method and system that recording is synthetic of realizing here on portable terminal, to realize that the recording complex functionality that provides on the portable terminal (mobile phone) is provided the user, the problem that can oneself record and dub for existing multimedia material.Below embodiment is described in detail, but not as a limitation of the invention.

Realize the concrete workflow of the whole functional of the method for the invention, as shown in Figure 1, concrete steps are as follows:

The first step provides recording synthetic function items in mobile phone menu, and corresponding shortcut perhaps is provided;

In second step, when entering the recording complex functionality, if do not insert earphone on the mobile phone, prompting needs earphone.If the insertion earphone, the manager that then opens a file therefrom can select to be intended for the multimedia file that it is dubbed;

The 3rd step, choosing will be for behind its multimedia file of dubbing on the portable terminal, voice data in the multimedia file and video data are split, the voice data and the video data that split the back generation are decoded, generate decoded video data and linear PCM (Pulse Code Modulation, pulse-code modulation recording) voice data of form and output begin to play this multimedia file;

The 4th step, during play multimedia files, the voice data of decoded video data and linear pulse-code modulation recording form is backuped in the internal memory of mobile phone (if the video format of the synthetic front and back of recording is identical here, then the data before the video decode are directly backed up as video, otherwise with the video data behind the video decode, carry out again being used as backup again after the respective coding); Begin recording simultaneously, recording is the voice data from audio frequency load module input PCM form;

The 5th step, the digital audio-frequency data that to import the decoded linear PCM form of the voice data of (recording) and output with the pcm audio superposition algorithm is synthesized together and encodes, then with information recompiles such as corresponding picture, video datas, and be stored in the temporary file;

In the 6th step, after multimedia file played, the user ejected edit box and waits for the user-in file name by preserving key, after the user-in file name, temporary file is renamed as the filename of input.If the user selects not preserve, then temporary file is deleted.

In the above-mentioned steps, data such as audio or video are accomplished to play back in decoding needs the regular hour (being assumed to be t1); And be input to and be converted into digital audio-frequency data from the audio frequency of simulation, and be input to the mobile phone master chip, also need the regular hour (being assumed to be t2).So when the audio or video of input audio frequency and output superposes, the audio or video data of current firm decoding and the voice data of positive input can not be superposeed, otherwise can cause the problem that recording misplaces in the final multimedia file that synthesizes.Should be with the voice data of current input and (t1+t2) output audio or the video superimpose of decoding before the time.

In the above-mentioned steps, the voice data of PCM (Pulse Code Modulation, the pulse-code modulation recording) form of input and output is superposeed when synthetic, the scale factor of input and output can all be 0.5.Further improve be by MMI (man-machine interface) for the user provides interface, select configuration by the cellphone subscriber.

Here can be used for to its multimedia file of dubbing can be the music of the various forms of mobile phone support, as midi, and mp3 etc.; Can be animation also, as gif, forms such as swf; Or video clip, as forms such as 3gpp, mp4.But precondition is the decodes and plays that mobile phone must be supported the multi-medium data of the type.The form of the multimedia file after dubbing is the form of mobile phone acquiescence, can be mp3, mp4, and 3gpp, mobile phones such as avi can support it is carried out the form of encoding and decoding.

Because when dubbing, need one side to play, the one side recording, and may itself be exactly voice data or animation and the video that has sound as the multimedia file of material.Here caused echo noise by recording again with original sound stack for the sound that prevents to play, before described method step, just before beginning, recording synthetic method of the present invention also requires to use earphone, if so under the situation of not inserting earphone, open the recording complex functionality, must eject the prompted dialog frame and also start timer, requiring the user to insert earphone can use, and does not then finish the step of this method and withdraws from otherwise do not insert earphone in the stipulated time.

As shown in Figure 2, be the synthetic system construction drawing of realization recording of the present invention.

Also need mobile phone that the synthetic menu item of recording or shortcut and relevant MMI (man-machine interface) are set in the system of the present invention, menu item or shortcut are in order to offer the cellphone subscriber, as recording complex functionality inlet; MMI is used for carrying out man-machine interaction; Here also need to be provided with judge module, be used to judge whether insert earphone on the portable terminal, then point out the user to insert earphone and start timer regularly, then do not finish to log off as not inserting earphone in the stipulated time of timer as not inserting earphone; Go into earphone if any the stipulated time interpolation of inserting earphone or described timer, then indicate 201 work of multi-media decoding and encoding module;

Multi-media decoding andencoding module 201 is used for will being its multimedia file encoding and decoding of dubbing to what choose on the mobile phone, merges and fractionation voice data and video data;

Audiocoding decoding module 202 is used for the voice data that multimedia file fractionation back generates is carried out audio decoder, and decoded data layout is the linear pcm digital audio format, and the voice data that export audiofrequency laminating module 207 stack backs is encoded;

Coding anddecoding video module 203 is used for carrying out the Code And Decode of video, specifically can decoding picture and data such as video; Mainly be that the video data that generates after multi-media decoding andencoding module 201 is split is decoded, to the coding video data ofvideo backup module 205 backup back outputs; If the video format before and after recording is synthetic is identical, then the data before the video decode is directly issuedvideo backup module 205, otherwise, carry out again issuingvideo backup module 205 again after the respective coding the video data behind the video decode;

Audiofrequency backup module 204 is used for the outputting audio data of decoded PCM form is backed up, and gives over to recording and synthesizes;

Video backup module 205 is used for front or rear video (picture) data of decoding are backed up, and gives over to recording and synthesizes;

Audio frequency laminating module (audio mixing) 207 is used for the voice data of the decoded linear pulse-code modulation recording form of audio coding decoding module and the voice data of recording are synthesized together;

Audiofrequency output module 209, outputting audio data is play voice and music;

Audiofrequency load module 206, input audio data is used for recording, generates the voice data of linear PCM form;

Video output module 208 is used for display video, animation and picture, output video coding/decoding module 203 decoded video datas.

System of the present invention begin recording synthetic after, at first read the multi-medium data that need dub from file system interface.Decode then, split the Voice ﹠ Video data,, then skip over this step if having only audio frequency or video in the material of multimedia file.Again audio frequency and/or video data are decoded.Digital audio-frequency data to the PCM behind the audio decoder (Pulse Code Modulation, pulse-code modulation recording) form is simultaneously exported broadcast, and one side backs up, and gives over to the synthetic usefulness of recording.Data behind the video decode are exported demonstration, but will if the video format after recording is synthesized is identical with the video format of former multimedia material, then can directly use the preceding video data of decoding in two kinds of situation as backup to the backup of video data; If the video format before and after recording is synthetic is inequality, then needs and to be used as backup behind the decoded coding video data again.Will record in play multimedia files, with voice data and the backup of the voice data of corresponding time that recording obtains, the audio mixing that superposes is encoded then.With the coding of the Composite tone data behind the coding with video data backup the carrying out multimedia file of corresponding time, form the multi-medium data after final recording is synthesized, deposit in the file system then.

Compared with prior art, the described method and system of the embodiment of the invention can add the audio that the user needs in original multimedia material to, so no longer only simple multimedia files such as playing back music of portable terminal (mobile phone), can also be used to recording synthetic multimedia, produce personalized the tinkle of bells and multimedia short-movie.

Certainly; the present invention also can have other various embodiments; under the situation that does not deviate from spirit of the present invention and essence thereof; those of ordinary skill in the art can make various corresponding changes and distortion according to the present invention, but these corresponding changes and distortion all should belong to the protection domain of the appended claim of the present invention.

Claims (9)

Translated fromChinese

1.一种在移动终端上实现录音合成的方法，应用于支持多媒体数据的移动终端中，其特征在于，包括以下步骤：1. A method for realizing recording synthesis on a mobile terminal, applied in a mobile terminal supporting multimedia data, is characterized in that, comprising the following steps:选择所述移动终端上要为其配音的多媒体文件，对所述多媒体文件进行音频和视频的拆分，对拆分后生成的音频数据和视频数据进行解码，生成解码后的视频数据和线性脉码调制录音格式的音频数据，并对该解码后的音频数据进行输出并备份，如果录音合成前后的视频格式相同，则将视频解码前的数据直接用作视频备份，否则将视频解码后的视频数据，再进行相应的编码后再用作备份；Select the multimedia file to be dubbed for it on the mobile terminal, split the audio and video of the multimedia file, decode the audio data and video data generated after the split, and generate decoded video data and linear pulse Code modulates the audio data in the recording format, and outputs and backs up the decoded audio data. If the video format before and after the recording synthesis is the same, the data before the video decoding is directly used as the video backup, otherwise the decoded video Data, and then encode accordingly before use as a backup;在所述移动终端上开始录音，生成线性脉码调制录音格式的音频数据，将该录制的音频数据和所述解码后的线性脉码调制录音格式的音频数据合成在一起并进行编码，对所述备份后的视频数据与所述合成编码后的音频数据进行合并编码，并输出。Start recording on the mobile terminal, generate audio data in the linear pulse code modulation recording format, synthesize and encode the recorded audio data and the decoded audio data in the linear pulse code modulation recording format, and encode the The backed up video data and the synthesized and coded audio data are merged and coded, and output.2.如权利要求1所述的方法，其特征在于，所述该录制的音频数据和所述解码后的线性脉码调制录音格式的音频数据合成在一起，进一步为运用脉码调制录音格式的音频叠加算法，将该录制的音频数据和所述解码后的线性脉码调制录音格式的音频数据合成在一起。2. The method according to claim 1, wherein the recorded audio data and the decoded audio data in the linear pulse code modulation recording format are combined together, and further use the audio data in the pulse code modulation recording format The audio superposition algorithm synthesizes the recorded audio data and the decoded audio data in the linear pulse code modulation recording format together.3.如权利要求1或2所述的方法，其特征在于，所述多媒体文件，包括所述移动终端上支持的各种格式的音乐、动画以及视频短片。3. The method according to claim 1 or 2, wherein the multimedia file includes music, animation and video clips in various formats supported by the mobile terminal.4.如权利要求1所述的方法，其特征在于，所述方法的步骤之前，还包括判断所述移动终端上是否插入耳机，如没有插入耳机则提示用户插入耳机并定时，如规定时间内不插入耳机则结束该方法的步骤并退出；如有插入耳机或规定时间内插入耳机，则继续执行下面步骤。4. The method according to claim 1, characterized in that, before the steps of the method, it also includes judging whether an earphone is inserted into the mobile terminal, and if no earphone is inserted, prompting the user to insert an earphone and timing, such as within a specified time If the earphone is not plugged in, the steps of the method are ended and exited; if the earphone is plugged in or within the specified time, the following steps are continued.5.一种在移动终端上实现录音合成的系统，应用于支持多媒体数据的移动终端中，其特征在于，包括多媒体编解码模块、音频编解码模块，视频编解码模块、音频备份模块、视频备份模块、音频输入模块、音频输出模块、视频输出模块、音频叠加模块，其中5. A system for realizing recording synthesis on a mobile terminal, applied to a mobile terminal supporting multimedia data, characterized in that it includes a multimedia codec module, an audio codec module, a video codec module, an audio backup module, and a video backup module module, audio input module, audio output module, video output module, audio overlay module, where所述多媒体编解码模块，用于对选择所述移动终端上要为其配音的多媒体文件中的音频数据和视频数据进行拆分和合并；The multimedia codec module is used for splitting and merging the audio data and video data in the multimedia file selected to be dubbed for it on the mobile terminal;所述音频编解码模块，用于对所述多媒体编解码模块拆分后生成的音频数据进行解码，对所述音频叠加模块叠加后输出的音频数据进行编码；The audio codec module is used to decode the audio data generated after the split of the multimedia codec module, and encode the audio data output after the superposition of the audio superimposition module;所述视频编解码模块，用于对所述多媒体编解码模块拆分后生成的视频数据进行解码，如果录音合成前后的视频格式相同，则将视频解码前的数据直接发给所述视频备份模块，否则将视频解码后的视频数据，再进行相应的编码后再发给所述视频备份模块；The video codec module is used to decode the video data generated after the split of the multimedia codec module, if the video format before and after recording synthesis is the same, the data before video decoding is directly sent to the video backup module , otherwise the video data after video decoding is sent to the video backup module after corresponding encoding;所述音频备份模块，用于对所述音频编解码模块输出的音频数据进行备份；The audio backup module is used to back up the audio data output by the audio codec module;所述视频备份模块，用于对所述多媒体编解码模块对多媒体数据拆分后输出的视频数据或所述视频编解码模块编码后的视频数据进行备份；The video backup module is used to back up the video data output by the multimedia codec module after splitting the multimedia data or the video data encoded by the video codec module;所述音频输入模块，用于接收录音数据，生成线性脉码调制录音格式的音频数据；The audio input module is used to receive recording data and generate audio data in linear pulse code modulation recording format;所述音频叠加模块，用于将所述音频输入模块生成的音频数据和所述音频编解码模块解码后的线性脉码调制录音格式的音频数据进行合成。The audio superposition module is used to synthesize the audio data generated by the audio input module and the audio data in the linear pulse code modulation recording format decoded by the audio codec module.6.如权利要求5所述的系统，其特征在于，所述音频叠加模块，进一步用于运用脉码调制录音格式的音频叠加算法，将所述音频输入模块生成的音频数据和所述音频编解码模块解码后的线性脉码调制录音格式的音频数据进行合成。6. The system according to claim 5, wherein the audio superposition module is further configured to use the audio superposition algorithm of the pulse code modulation recording format to combine the audio data generated by the audio input module with the audio code The audio data in the linear pulse code modulation recording format decoded by the decoding module is synthesized.7.如权利要求5所述的系统，其特征在于，进一步包括：7. The system of claim 5, further comprising:音频输出模块，用于输出所述音频编解码模块解码后的音频数据；an audio output module, configured to output audio data decoded by the audio codec module;视频输出模块，用于输出所述视频编解码模块解码后的视频数据。The video output module is used to output the video data decoded by the video codec module.8.如权利要求5、6或7所述的系统，其特征在于，所述多媒体文件，包括所述移动终端上支持的各种格式的音乐、动画以及视频短片。8. The system according to claim 5, 6 or 7, wherein the multimedia files include music, animation and video clips in various formats supported by the mobile terminal.9.如权利要求5所述的系统，其特征在于，进一步包括一判断模块和定时器，其中，所述判断模块，用于判断所述移动终端上是否插入耳机，如没有插入耳机则提示用户插入耳机并启动所述定时器定时，如所述定时器的规定时间内不插入耳机则结束退出系统；如有插入耳机或所述定时器的规定时间内插入耳机，则指示所述多媒体编解码模块工作。9. The system according to claim 5, further comprising a judging module and a timer, wherein the judging module is used to judge whether an earphone is inserted into the mobile terminal, and if no earphone is inserted, the user is prompted Insert the earphone and start the timer timing. If the earphone is not inserted within the specified time of the timer, the system will end and exit the system; if the earphone is inserted or the earphone is inserted within the specified time of the timer, the multimedia codec will be instructed The module works.

Images