CN107241646B - Multimedia video editing method and device - Google Patents
Multimedia video editing method and deviceDownload PDFInfo
- Publication number
- CN107241646B CN107241646BCN201710566432.2ACN201710566432ACN107241646BCN 107241646 BCN107241646 BCN 107241646BCN 201710566432 ACN201710566432 ACN 201710566432ACN 107241646 BCN107241646 BCN 107241646B
- Authority
- CN
- China
- Prior art keywords
- video
- data
- image
- audio
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及多媒体技术领域，特别是涉及一种多媒体视频的编辑方法及装置。The present invention relates to the field of multimedia technology, and in particular, to a method and device for editing a multimedia video.
背景技术Background technique
随着互联网技术的快速发展，人们已经不再满足于单纯的使用手机通话来进行交流及沟通，其中，在线直播、小视频等使用多媒体技术建立的社交平台已经成为用户之间进行沟通的主要手段。With the rapid development of Internet technology, people are no longer satisfied with simply using mobile phone calls to communicate and communicate. Among them, online live broadcasts, small videos and other social platforms established by multimedia technology have become the main means of communication between users .
目前，用户在使用终端设备进行直播或录制小视频时，可以通过截取视频中的一小段进行保存，例如，某直播平台正在直播小女孩跳舞，为了记录小女孩旋转的视频，需要截取直播视频中小女孩旋转的短视频。在截取视频后，为了增强对视频内容的播放效果，对多媒体视频进行编辑已经成为亟待解决的问题。At present, when users use terminal devices to live broadcast or record small videos, they can save a small clip of the video. For example, a live broadcast platform is broadcasting a little girl dancing. Short video of girl spinning. After the video is intercepted, in order to enhance the playback effect of the video content, editing the multimedia video has become an urgent problem to be solved.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明提供一种多媒体视频的编辑方法及装置，主要目的在于现有直播或小视频中截取的短视频无法编辑的问题。In view of this, the present invention provides a method and device for editing a multimedia video, mainly aiming at the problem that the short video clipped from the existing live broadcast or small video cannot be edited.
依据本发明一个方面，提供了一种多媒体视频的编辑方法，包括：According to one aspect of the present invention, a method for editing a multimedia video is provided, comprising:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
进一步地，所述对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理包括：Further, performing rendering processing on the video data and performing audio track processing on the audio data includes:
接收用户输入的处理指令，所述处理指令中携带有效果标识；receiving a processing instruction input by the user, where the processing instruction carries an effect identifier;
根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。The video data is rendered according to the video effect identification in the effect identification, and the audio data is processed according to the audio effect identification in the effect identification.
进一步地，所述根据所述效果标识中的视频效果标识渲染所述视频数据包括：Further, the rendering of the video data according to the video effect identifier in the effect identifier includes:
提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；Extracting the image data of each frame in the video data, and performing filter processing on the image data;
根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。The target image in the filter-processed image data is identified according to the video effect identifier, and the target image is synthesized and rendered.
进一步地，所述根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染包括：Further, identifying the target image in the image data after filter processing according to the video effect identifier, and performing composite rendering on the target image includes:
若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。If it is identified that the video effect is identified as a composite stereoscopic image, the target image is segmented, and the target image, the segmented target image and the rendered image are colored and synthesized according to a preset coloring rule, and the preset coloring is performed. The rule is used to reflect the position display relationship among the target image, the segmented target image, and the rendered image.
进一步地，所述根据所述效果标识中的音频效果标识处理所述音频数据包括：Further, the processing of the audio data according to the audio effect identifier in the effect identifier includes:
按照预设时间间隔采集所述音频数据中的离散音轨数据；Collect discrete audio track data in the audio data according to preset time intervals;
根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。The discrete audio track data and the preset audio track are effectively superimposed according to the audio effect identifier.
进一步地，所述解码所述多媒体文件中的视频数据及音频数据包括：Further, the decoding of the video data and audio data in the multimedia file includes:
按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。The video data and audio data in the multimedia file are decoded according to the video track and the audio track, respectively.
进一步地，所述对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理之后，所述方法还包括：Further, after the rendering processing is performed on the video data and the audio track processing is performed on the audio data, the method further includes:
当接收到实时预览请求时，展示所述视频数据及所述音频数据。The video data and the audio data are displayed when a real-time preview request is received.
进一步地，所述方法还包括：Further, the method also includes:
接收速度调整指令，根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。A speed adjustment instruction is received, and the playback speed of video data and audio data in the multimedia video is adjusted according to the speed information carried in the speed adjustment instruction.
依据本发明一个方面，提供了一种多媒体视频的编辑装置，包括：According to one aspect of the present invention, a multimedia video editing device is provided, comprising:
获取单元，用于获取多媒体文件；an acquisition unit for acquiring multimedia files;
解码单元，用于解码所述多媒体文件中的视频数据及音频数据；a decoding unit for decoding video data and audio data in the multimedia file;
处理单元，用于对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；a processing unit, configured to perform rendering processing on the video data and track processing on the audio data;
编码单元，用于将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The encoding unit is used for encoding the processed video data and the processed audio data to obtain a multimedia video.
进一步地，所述处理单元包括：Further, the processing unit includes:
接收模块，用于接收用户输入的处理指令，所述处理指令中携带有效果标识；a receiving module, configured to receive a processing instruction input by a user, wherein the processing instruction carries an effect identifier;
处理模块，用于根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。A processing module, configured to render the video data according to the video effect identification in the effect identification, and process the audio data according to the audio effect identification in the effect identification.
进一步地，所述处理模块包括：Further, the processing module includes:
提取子模块，用于提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；Extraction submodule for extracting the image data of each frame in the video data, and performing filter processing on the image data;
合成子模块，用于根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。A synthesis sub-module, configured to identify a target image in the image data after filter processing according to the video effect identifier, and perform synthesis and rendering on the target image.
所述合成子模块，具体用于若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。The synthesizing sub-module is specifically configured to segment the target image if it is identified that the video effect is identified as a synthesizing stereoscopic image, and perform the segmentation of the target image, the segmented target image and the rendered image according to a preset coloring rule. Perform coloring synthesis, and the preset coloring rule is used to reflect the position display relationship among the target image, the segmented target image, and the rendered image.
进一步地，所述处理模块还包括：Further, the processing module also includes:
采集子模块，用于按照预设时间间隔采集所述音频数据中的离散音轨数据；a collection sub-module for collecting discrete audio track data in the audio data according to preset time intervals;
叠加子模块，用于根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。A superimposition sub-module, configured to effectively superimpose the discrete audio track data and the preset audio track according to the audio effect identifier.
所述解码单元，具体用于按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。The decoding unit is specifically configured to decode the video data and the audio data in the multimedia file according to the video track and the audio track respectively.
进一步地，所述装置还包括：Further, the device also includes:
展示单元，用于当接收到实时预览请求时，展示所述视频数据及所述音频数据。The display unit is configured to display the video data and the audio data when a real-time preview request is received.
进一步地，所述装置还包括：Further, the device also includes:
调整单元，用于接收速度调整指令，并根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。The adjustment unit is configured to receive a speed adjustment instruction, and adjust the playback speed of video data and audio data in the multimedia video according to the speed information carried in the speed adjustment instruction.
依据本发明一个方面，提供了一种存储设备，其中存储有多条指令，所述指令适于由处理器加载并执行：According to one aspect of the present invention, there is provided a storage device in which a plurality of instructions are stored, the instructions are adapted to be loaded and executed by a processor:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
依据本发明一个方面，提供了一种移动终端，包括处理器，适于实现各种指令；以及存储设备，适于存储多条指令，所述指令适于由处理器加载并执行：According to one aspect of the present invention, there is provided a mobile terminal, comprising a processor adapted to implement various instructions; and a storage device adapted to store a plurality of instructions, the instructions adapted to be loaded and executed by the processor:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
借由上述技术方案，本发明实施例提供的技术方案至少具有下列优点：With the above technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:
本发明提供了一种多媒体视频的编辑方法及装置，首先获取多媒体文件，然后解码所述多媒体文件中的视频数据及音频数据，再对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理，最后将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。与现有直播或小视频中截取的短视频无法编辑相比，本发明实施例通过解码出多媒体文件中的视频数据和音频数据，分别对视频数据和音频数据进行处理，在编码为多媒体视频，实现对直播或截取的视频进行编辑，增加短视频的播放效果，使得视频更加生动，编辑后视频中的人物与渲染图像更加贴合，提高视频的使用效率。The present invention provides a multimedia video editing method and device. First, a multimedia file is acquired, then video data and audio data in the multimedia file are decoded, and then the video data is rendered and processed. The audio track is processed, and finally the processed video data and the processed audio data are encoded to obtain a multimedia video. Compared with the short video intercepted from the existing live broadcast or small video, which cannot be edited, the embodiment of the present invention processes the video data and the audio data respectively by decoding the video data and the audio data in the multimedia file, and encodes them into a multimedia video. Realize the editing of live or intercepted video, increase the playback effect of short video, make the video more vivid, and the characters in the edited video fit more closely with the rendered image, improving the efficiency of video usage.
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.
附图说明Description of drawings
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:
图1示出了本发明实施例一提供的一种多媒体视频的编辑方法流程图；1 shows a flowchart of a method for editing a multimedia video according to Embodiment 1 of the present invention;
图2示出了本发明实施例二提供的另一种多媒体视频的编辑方法流程图；2 shows a flowchart of another method for editing a multimedia video provided by Embodiment 2 of the present invention;
图3示出了本发明实施例三提供的一种多媒体视频的编辑装置框图；3 shows a block diagram of an apparatus for editing a multimedia video provided by Embodiment 3 of the present invention;
图4示出了本发明实施例四提供的另一种多媒体视频的编辑装置框图。FIG. 4 shows a block diagram of another apparatus for editing a multimedia video according to Embodiment 4 of the present invention.
具体实施方式Detailed ways
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.
本发明实施例提供了一种多媒体视频的编辑方法，如图1所示，所述方法包括：An embodiment of the present invention provides a method for editing a multimedia video, as shown in FIG. 1 , the method includes:
101、获取多媒体文件。101. Acquire a multimedia file.
其中，所述多媒体文件可以为不同格式的视频文件，如MP4格式、MKV格式、3GP格式等，本发明实施例不做具体限定，所述多媒体文件可以由终端设备中的摄像头进行拍摄获取，也可以由在线直播的视频中进行截取，还可以直接从终端设备中的存储空间中进行提取，本发明实施例不做具体限定。The multimedia files may be video files in different formats, such as MP4 format, MKV format, 3GP format, etc., which are not specifically limited in this embodiment of the present invention, and the multimedia files may be captured by a camera in a terminal device, or It can be intercepted from the video of the online live broadcast, and can also be directly extracted from the storage space in the terminal device, which is not specifically limited in the embodiment of the present invention.
需要说明的是，为了便于多媒体视频的编辑应用到终端设备中，在截取视频或录制视频时需要设置一定的视频播放时间，使得最终生成的多媒体视频为一个较短的视频，便于当前的视频编辑方法应用到内存空间较小的终端设备中。It should be noted that, in order to facilitate the application of multimedia video editing to terminal devices, a certain video playback time needs to be set when intercepting or recording videos, so that the final generated multimedia video is a short video, which is convenient for current video editing. The method is applied to terminal devices with small memory space.
102、解码所述多媒体文件中的视频数据及音频数据。102. Decode video data and audio data in the multimedia file.
其中，所述解码视频数据及音频数据具体可以为通过分别读取多媒体文件中的视频数据及音频数据来实现视频及音频的解码，即将视频流及音频流解码还原成模拟视频数据及模拟音频数据。Wherein, the described decoding video data and audio data can be realized by reading the video data and audio data in the multimedia file respectively to realize the decoding of video and audio, that is, the decoding of the video stream and audio stream is restored to analog video data and analog audio data. .
需要说明的是，在本发明实施例中，解码过程可以通过一个媒体解码器完成，将多媒体文件输送至媒体解码器中，可以自动得到视频数据及音频时间，对于视频数据而言，由于视频是由多帧不同的图像组合而成的，解码后的视频数据可以具体为每一帧对应的图像信息，对于音频数据而言，解码后的音频数据则为脉冲形式的模拟信号。It should be noted that, in this embodiment of the present invention, the decoding process can be completed by a media decoder, and the multimedia file is sent to the media decoder, and the video data and audio time can be automatically obtained. Composed of multiple frames of different images, the decoded video data may specifically be image information corresponding to each frame, and for audio data, the decoded audio data is an analog signal in the form of pulses.
103、对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理。103. Perform rendering processing on the video data, and perform audio track processing on the audio data.
其中，所述渲染处理包括在视频中添加位图、添加动态图像、调整视频图像的图像效果等，音轨处理包括增加不同音轨进行组合、增加音效等，本发明实施例不做具体限定，位图即为点阵图像或绘制图像。The rendering processing includes adding a bitmap to the video, adding a dynamic image, adjusting the image effect of the video image, etc., and the audio track processing includes adding different audio tracks for combination, adding sound effects, etc., which are not specifically limited in the embodiment of the present invention. A bitmap is a bitmap or drawn image.
需要说明的是，在视频数据及音频数据进行处理时，可以单独分开进行处理，也可以相互关联着进行处理。例如，在对视频中添加一个背景时，可以只对视频数据进行渲染背景处理，而不对音轨数据进行处理，但是，视频中人物吐字时，想要将吐出的字转变为文字添加在视频中，就需要先处理音轨数据，根据识别音轨数据中的文字，将文字库中对应的图像数据添加在视频数据中，这时就需要视频数据与音轨数据共同处理。It should be noted that, when the video data and the audio data are processed, they may be processed separately, or may be processed in association with each other. For example, when adding a background to a video, you can only perform background rendering processing on the video data without processing the audio track data. However, when the characters in the video spit out words, you want to convert the spit out words into text and add them in the video , the audio track data needs to be processed first, and the corresponding image data in the text library is added to the video data according to the characters in the recognized audio track data. At this time, the video data and the audio track data need to be processed together.
104、将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。104. Encode the processed video data and the processed audio data to obtain a multimedia video.
其中，所述编码为将渲染后的视频数据与音频数据进行匹配的编码，以得到流畅、音视频对应的多媒体视频。The encoding is encoding for matching the rendered video data with the audio data, so as to obtain a smooth multimedia video corresponding to the audio and video.
本发明提供了一种多媒体视频的编辑方法，与现有直播或小视频中截取的短视频无法编辑相比，本发明实施例通过解码出多媒体文件中的视频数据和音频数据，分别对视频数据和音频数据进行处理，在编码为多媒体视频，实现对直播或截取的视频进行编辑，增加短视频的播放效果，使得视频更加生动，编辑后视频中的人物与渲染图像更加贴合，提高视频的使用效率。The present invention provides a method for editing a multimedia video. Compared with the existing short video clipped from a live broadcast or a small video, which cannot be edited, the embodiment of the present invention decodes the video data and audio data in the multimedia file, respectively, for the video data. It is processed with audio data, and encoded into multimedia video, to realize the editing of live or intercepted video, increase the playback effect of short video, make the video more vivid, and the characters in the edited video fit more closely with the rendered image, improving the quality of the video. Use efficiency.
本发明实施例提供了另一种多媒体视频的编辑方法，如图2所示，所述方法包括：An embodiment of the present invention provides another method for editing a multimedia video. As shown in FIG. 2 , the method includes:
201、获取多媒体文件。201. Acquire a multimedia file.
本步骤与图1所示的步骤101方法相同，在此不再赘述。This step is the same as the method of
需要说明的是，本发明实施例中涉及的多媒体视频的编辑方法可以应用于其他直播或录制视频的应用程序中，通过调用接口来实现视频的编辑，还可以根据对应的程序来编写为单独使用的应用程序，通过调用摄像头直接拍摄得到多媒体文件，本发明实施例不做具体限定。It should be noted that the multimedia video editing method involved in the embodiment of the present invention can be applied to other application programs for live broadcasting or recording video, and the editing of the video can be realized by calling the interface, and can also be written according to the corresponding program for independent use. The application program obtained by invoking the camera to directly shoot the multimedia file, which is not specifically limited in the embodiment of the present invention.
202、按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。202. Decode the video data and audio data in the multimedia file according to the video track and the audio track, respectively.
其中，所述视频轨迹为视频播放的内容轨迹，所述音频轨迹为音频播放的内容轨迹，为了解码出多媒体文件中的视频数据及音轨数据，以便对视频数据及音轨数据分别进行处理，因此需要按照视频轨迹与音频轨迹分别进行解码。Wherein, the video track is the content track of the video playback, and the audio track is the content track of the audio playback, in order to decode the video data and the audio track data in the multimedia file, so as to process the video data and the audio track data respectively, Therefore, it needs to be decoded separately according to the video track and the audio track.
203、接收用户输入的处理指令。203. Receive a processing instruction input by a user.
其中，所述处理指令中携带有效果标识。所示处理指令用于指示系统进行视频具体的视频编辑，所示效果标识为标识可以达到不同视频及音频效果的信息，例如，渲染图像、语音文字转换、添加背景等，本发明实施例不做具体限定。Wherein, the processing instruction carries an effect identifier. The processing instruction shown is used to instruct the system to perform specific video editing of the video, and the effect identification is to identify information that can achieve different video and audio effects, for example, rendering an image, converting voice to text, adding a background, etc. Specific restrictions.
需要说明的是，若渲染的图像为用户输入的图像，则可以将图像通过处理指令进行传入。另外，需要渲染的图像、添加的背景等位图可以为预先设置好的图像，也可以为用户进行输入的图像，本发明实施例不做具体限定。It should be noted that, if the rendered image is an image input by the user, the image can be passed in through a processing instruction. In addition, bitmaps such as the image to be rendered and the background to be added may be preset images or images input by the user, which are not specifically limited in this embodiment of the present invention.
204、根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。204. Render the video data according to the video effect identification in the effect identification, and process the audio data according to the audio effect identification in the effect identification.
其中，所述视频效果标识为针对在视频数据中进行处理视频效果的标识，所述音频效果标识为针对在音频数据中进行处理音频效果的标识，为了进一步的添加不同效果对应的不同图像，需要根据视频或音频效果标识来处理视频数据或音频数据。Wherein, the video effect identifier is an identifier for processing video effects in video data, and the audio effect identifier is an identifier for processing audio effects in audio data. In order to further add different images corresponding to different effects, it is necessary to The video data or the audio data is processed according to the video or audio effect identification.
通过根据效果标识中的视频效果标识及音频效果标识分别对视频及音频进行渲染处理及音效处理，使得图像与声音分别进行编辑，优化对视频编辑的性能。By performing rendering processing and sound effect processing on the video and audio respectively according to the video effect identifier and the audio effect identifier in the effect identifier, the image and the sound are edited separately, and the performance of the video editing is optimized.
对于本发明实施例，步骤根据所述效果标识中的视频效果标识渲染所述视频数据具体可以包括：提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。For the embodiment of the present invention, the step of rendering the video data according to the video effect identifier in the effect identifier may specifically include: extracting image data of each frame in the video data, and performing filter processing on the image data; The target image in the filter-processed image data is identified according to the video effect identifier, and the target image is synthesized and rendered.
其中，由于解析后的视频数据是由一帧一帧的图像信息组成的，为了在视频中添加图像，需要对每一帧中的图像信息添加图像，而在对图像信息进行处理之前，需要进行滤镜处理，从而得到需要过滤后的视频效果。所述目标图像为需要添加位图的对应，或者为需要进行渲染的对象，本发明实施例不做具体限定，例如，当视频效果标识为添加背景图像，则目标图像则为人物图像或动物图像，若视频效果标识为添加吐字特效，则目标图像为人脸或人嘴。Among them, since the parsed video data is composed of frame-by-frame image information, in order to add an image to the video, it is necessary to add an image to the image information in each frame, and before processing the image information, it is necessary to Filter processing, so as to obtain the video effect that needs to be filtered. The target image is a corresponding bitmap that needs to be added, or an object that needs to be rendered, which is not specifically limited in this embodiment of the present invention. For example, when the video effect is identified as adding a background image, the target image is a human image or an animal image. , if the video effect is marked as adding special effects, the target image is a human face or a human mouth.
需要说明的是，合成渲染则为在每一帧图像中添加需要添加的位图，每一帧中添加位图的位置均不同，从而实现视频播放中的渲染的图像为动态的。It should be noted that the composite rendering is to add the bitmap to be added in each frame of image, and the position of adding the bitmap in each frame is different, so that the rendered image in the video playback is dynamic.
对于本发明实施例，步骤根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染具体可以包括：若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。For the embodiment of the present invention, the step of identifying the target image in the filtered image data according to the video effect identifier, and performing composite rendering on the target image may specifically include: if the video effect identifier is identified as a composite stereoscopic image , the target image is segmented, and the target image, the segmented target image and the rendered image are colored and synthesized according to a preset coloring rule, and the preset coloring rule is used to reflect the target image, the segmentation The position display relationship between the target image after and the rendered image.
其中，所述合成立体图像为利用视觉差异效果将位图展示为带有层次感的、虚拟现实立体动态的图像，这种立体图像取决于添加的位图在每一帧图像中不同的位置是否显示，显示多少，从而得到的。Wherein, the synthetic stereoscopic image is to use the visual difference effect to display the bitmap as a layered, virtual reality stereoscopic dynamic image, and this stereoscopic image depends on whether the added bitmap is in different positions in each frame of image. Show, show how much, and thus get.
需要说明的是，若视频效果标识为合成立体图像，具体的步骤则为分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，其中，一般的，合成立体图像的目标图像为人物图像，为了将图像中的人物与背景进行区分，需要对每一帧的图像信息进行分割，所述渲染图像为需要添加的位图，所述预置着色规则是判断渲染图像覆盖目标图像时是否显示、显示多少，以及渲染图像是否需要隐藏的策略，具体策略根据不同位图及人物的位置进行设定，本发明实施例不做具体限定。It should be noted that, if the video effect is identified as synthesizing a stereoscopic image, the specific step is to segment the target image, and perform coloring and synthesizing on the target image, the segmented target image and the rendered image according to a preset coloring rule, Wherein, generally, the target image for synthesizing the stereoscopic image is a person image. In order to distinguish the person in the image from the background, the image information of each frame needs to be segmented, and the rendered image is a bitmap that needs to be added. The preset coloring rule is a strategy for judging whether the rendered image is displayed when covering the target image, how much to display, and whether the rendered image needs to be hidden.
对于本发明实施例，步骤根据所述效果标识中的音频效果标识处理所述音频数据具体可以包括：按照预设时间间隔采集所述音频数据中的离散音轨数据；根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。For the embodiment of the present invention, the step of processing the audio data according to the audio effect identifier in the effect identifier may specifically include: collecting discrete audio track data in the audio data according to a preset time interval; The discrete audio track data is effectively superimposed with the preset audio track.
为了更好的将不同音轨进行叠加，而不是单单是音量进行简单的叠加，需要对音频数据进行离散化，按照预设时间间隔采集离散的音轨数据，所述预设时间间隔可以为1秒、0.05秒等，本发明实施例不做具体限定。所述有效叠加可以为将多个预设音轨的离散音轨数据进行叠加，除了多媒体文件中的音频数据采集的离散音轨数据，其他的预设音轨可以为存储在当前终端设备的缓存中或硬盘中，本发明实施例不做具体限定。例如，采集的离散音轨数据为小朋友读诗的声音，需要重叠添加的预设音轨为荷塘月色的背景音乐，则将声音进行重叠叠加。In order to better superimpose different audio tracks, instead of simply superimposing the volume, it is necessary to discretize the audio data, and collect the discrete audio track data according to a preset time interval, and the preset time interval can be 1 seconds, 0.05 seconds, etc., which are not specifically limited in the embodiment of the present invention. The effective superposition may be to superimpose the discrete audio track data of a plurality of preset audio tracks, except for the discrete audio track data collected by the audio data in the multimedia file, other preset audio tracks may be stored in the cache of the current terminal device. It is not specifically limited in this embodiment of the present invention. For example, the collected discrete audio track data is the sound of children reading poetry, and the preset audio track that needs to be superimposed is the background music of the moonlight in the lotus pond, then the audio is superimposed and superimposed.
需要说明的是，在音频处理时，可以选择开源的音频处理方法，如Ffmpg。It should be noted that during audio processing, an open source audio processing method, such as Ffmpg, can be selected.
205、当接收到实时预览请求时，展示所述视频数据及所述音频数据。205. Display the video data and the audio data when a real-time preview request is received.
其中，所述实时预览请求为用户输入的需要预览当前处理视频或音频的状态的请求，实时预览请求用于指示模拟播放处理后的视频图像，可以为每一帧进行浏览，也可以以视频形式进行播放，还模拟播放处理后的音频，一般的，实时浏览请求还用于指示模拟展示未处理的原始图像及原始音频，具体由接收到实时预览请求的时间而定，本发明实施例不做具体限定。The real-time preview request is a request input by the user to preview the state of the currently processed video or audio, and the real-time preview request is used to instruct the simulated playback of the processed video image, which can be browsed for each frame or in the form of a video Play, and also simulate playback of the processed audio. Generally, the real-time browsing request is also used to instruct the simulation to display the unprocessed original image and original audio, which is specifically determined by the time when the real-time preview request is received. Specific restrictions.
206、将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。206. Encode the processed video data and the processed audio data to obtain a multimedia video.
本步骤与图1所示的步骤104方法相同，在此不再赘述。This step is the same as the method of
207、接收速度调整指令，根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。207. Receive a speed adjustment instruction, and adjust the playback speed of video data and audio data in the multimedia video according to the speed information carried in the speed adjustment instruction.
其中，所述速度调整指令中携带有速度信息，可以为加快速度或减慢速度，具体的数据可以由速度信息中携带，本发明实施例不做具体限定。The speed adjustment instruction carries speed information, which may be to speed up or slow down, and specific data may be carried in the speed information, which is not specifically limited in this embodiment of the present invention.
需要说明的是，对于速度的调整具体方法可以为若对于视频数据则可以调整1秒钟内图像的帧数，以实现调节播放视频的快慢速度，若对于音频数据则可以调整预设时间内播放音轨的快慢，以实现调节播放音频的快慢速度。It should be noted that the specific method for adjusting the speed can be: for video data, you can adjust the number of frames of the image within 1 second to adjust the speed of playing the video, and for audio data, you can adjust the playback within a preset time. The speed of the audio track to adjust the speed of the audio playback.
对于本发明实施例，具体的应用场景可以如下所示，但不限于此，包括：截取男孩在线读书的多媒体文件，按照视频轨迹及音频轨迹解码出男孩读书的视频数据及音频数据，用户输入的效果标识为吐字转换标识，则分别处理视频数据及音频数据，首先识别音频数据中的男孩读出的汉字，从预置文字库中提取对应的文字图像，预置文字库中存储有与文字语音对应的文字图像，将文字图像添加至视频数据中，即找到“锄禾日当午”的渲染图像或文字萌图，按照音频播放时间，将“锄”、“禾”、“日”、“当”、“午”分别添加至时间对应的帧的图像中，添加的位置为识别的男孩人脸，然后将添加文字图像及音频进行编码，得到编辑后的短视频。For the embodiment of the present invention, a specific application scenario may be as follows, but is not limited to this, including: intercepting a multimedia file of a boy reading online, decoding the video data and audio data of the boy reading according to the video track and audio track, and the user input The effect is marked as a character transformation mark, then the video data and the audio data are processed respectively, first, the Chinese characters read by the boy in the audio data are identified, and the corresponding text images are extracted from the preset text library, and the preset text library is stored. For the corresponding text image, add the text image to the video data, that is, find the rendered image or text cute picture of "The day of hoeing and hoeing". Dang" and "Noon" are respectively added to the image of the frame corresponding to the time, and the added position is the recognized boy's face, and then the added text image and audio are encoded to obtain the edited short video.
本发明提供了另一种多媒体视频的编辑方法，本发明实施例通过解码出多媒体文件中的视频数据和音频数据，根据视频效果标识对视频数据进行渲染，根据音频效果标识对音频数据进行有效叠加，再编码为多媒体视频，实现对直播或截取的视频进行编辑，增加短视频的播放效果，使得视频更加生动，提高视频内容的展现效果，编辑后视频中的人物与渲染图像更加贴合，可以根据不同的要求进行设计录制的短视频，增加短视频的用途，提高视频的使用效率。The present invention provides another multimedia video editing method. In the embodiment of the present invention, the video data and audio data in the multimedia file are decoded, the video data is rendered according to the video effect identifier, and the audio data is effectively superimposed according to the audio effect identifier. , and then encode it into a multimedia video to edit the live or intercepted video, increase the playback effect of the short video, make the video more vivid, and improve the display effect of the video content. Design and record short videos according to different requirements, increase the use of short videos, and improve the efficiency of video usage.
进一步的，作为对上述图1所示方法的实现，本发明实施例提供了一种多媒体视频的编辑装置，如图3所示，该装置包括：获取单元31、解码单元32、处理单元33、编码单元34。Further, as an implementation of the method shown in FIG. 1, an embodiment of the present invention provides a multimedia video editing device. As shown in FIG. 3, the device includes: an
获取单元31，用于获取多媒体文件；所述获取单元31为多媒体视频的编辑装置执行获取多媒体文件的功能模块。The obtaining
解码单元32，用于解码所述多媒体文件中的视频数据及音频数据；所述解码单元32为多媒体视频的编辑装置执行解码所述多媒体文件中的视频数据及音频数据的功能模块。The
处理单元33，用于对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；所述处理单元33为多媒体视频的编辑装置执行对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理的功能模块。The
编码单元34，用于将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。所述编码单元34为多媒体视频的编辑装置执行将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频的功能模块。The
本发明提供了一种多媒体视频的编辑装置，与现有直播或小视频中截取的短视频无法编辑相比，本发明实施例通过解码出多媒体文件中的视频数据和音频数据，分别对视频数据和音频数据进行处理，在编码为多媒体视频，实现对直播或截取的视频进行编辑，增加短视频的播放效果，使得视频更加生动，编辑后视频中的人物与渲染图像更加贴合，提高视频的使用效率。The present invention provides a multimedia video editing device. Compared with the short video clipped from the existing live broadcast or small video, which cannot be edited, the embodiment of the present invention decodes the video data and audio data in the multimedia file, respectively, for the video data. It is processed with audio data, and encoded into multimedia video, to realize the editing of live or intercepted video, increase the playback effect of short video, make the video more vivid, and the characters in the edited video fit more closely with the rendered image, improving the quality of the video. Use efficiency.
进一步的，作为对上述图2所示方法的实现，本发明实施例提供了另一种多媒体视频的编辑装置，如图4所示，该装置包括：获取单元41、解码单元42、处理单元43、编码单元44、展示单元45、调整单元46。Further, as an implementation of the method shown in FIG. 2 above, an embodiment of the present invention provides another apparatus for editing multimedia video. As shown in FIG. 4 , the apparatus includes: an
获取单元41，用于获取多媒体文件；an
解码单元42，用于解码所述多媒体文件中的视频数据及音频数据；Decoding
处理单元43，用于对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；a
编码单元44，用于将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The
具体的，为了便于根据用户的需求进行处理视频及音频，所述处理单元43包括：Specifically, in order to facilitate the processing of video and audio according to user requirements, the
接收模块4301，用于接收用户输入的处理指令，所述处理指令中携带有效果标识；A
处理模块4302，用于根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。The
具体的，为了具体实现对视频数据的处理步骤，所述处理模块4302包括：Specifically, in order to specifically implement the processing steps for video data, the
提取子模块430201，用于提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；
合成子模块430202，用于根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。The
所述合成子模块430202，具体用于若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。The synthesizing
具体的，为了具体实现对音频数据的处理步骤，所述处理模块4302还包括：Specifically, in order to specifically implement the processing steps for audio data, the
采集子模块430203，用于按照预设时间间隔采集所述音频数据中的离散音轨数据；The
叠加子模块430204，用于根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。The
所述解码单元42，具体用于按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。The
进一步地，为了便于用户随时进行预览渲染的视频及处理的音频，所述装置还包括：Further, in order to facilitate the user to preview the rendered video and the processed audio at any time, the device further includes:
展示单元45，用于当接收到实时预览请求时，展示所述视频数据及所述音频数据。The
进一步地，为了可以随意调整播放视频的速度，所述装置还包括：Further, in order to be able to adjust the speed of playing the video at will, the device further includes:
调整单元46，用于接收速度调整指令，并根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。The
本发明提供了另一种多媒体视频的编辑装置，本发明实施例通过解码出多媒体文件中的视频数据和音频数据，根据视频效果标识对视频数据进行渲染，根据音频效果标识对音频数据进行有效叠加，再编码为多媒体视频，实现对直播或截取的视频进行编辑，增加短视频的播放效果，使得视频更加生动，提高视频内容的展现效果，编辑后视频中的人物与渲染图像更加贴合，可以根据不同的要求进行设计录制的短视频，增加短视频的用途，提高视频的使用效率。The present invention provides another multimedia video editing device. In the embodiment of the present invention, the video data and audio data in the multimedia file are decoded, the video data is rendered according to the video effect identifier, and the audio data is effectively superimposed according to the audio effect identifier. , and then encode it into a multimedia video to edit the live or intercepted video, increase the playback effect of the short video, make the video more vivid, and improve the display effect of the video content. Design and record short videos according to different requirements, increase the use of short videos, and improve the efficiency of video usage.
本发明实施例提供了一种存储设备，其中存储有多条指令，所述指令适于由处理器加载并执行：获取多媒体文件；解码所述多媒体文件中的视频数据及音频数据；对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。An embodiment of the present invention provides a storage device, in which a plurality of instructions are stored, and the instructions are suitable for being loaded and executed by a processor: acquiring a multimedia file; decoding video data and audio data in the multimedia file; Rendering processing is performed on the video data, and audio track processing is performed on the audio data; the processed video data and the processed audio data are encoded to obtain a multimedia video.
本发明实施例提供了一种移动终端，包括处理器，适于实现各种指令；以及存储设备，适于存储多条指令，所述指令适于由处理器加载并执行：获取多媒体文件；解码所述多媒体文件中的视频数据及音频数据；对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。An embodiment of the present invention provides a mobile terminal, including a processor, which is suitable for implementing various instructions; and a storage device, which is suitable for storing a plurality of instructions, and the instructions are suitable for being loaded and executed by the processor: acquiring a multimedia file; decoding video data and audio data in the multimedia file; rendering processing on the video data, and performing audio track processing on the audio data; encoding the processed video data and the processed audio data to obtain a multimedia video .
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.
可以理解的是，上述方法及装置中的相关特征可以相互参考。另外，上述实施例中的“第一”、“第二”等是用于区分各实施例，而并不代表各实施例的优劣。It can be understood that the relevant features in the above-mentioned methods and apparatuses may refer to each other. In addition, "first", "second", etc. in the above-mentioned embodiments are used to distinguish each embodiment, and do not represent the advantages and disadvantages of each embodiment.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.
在此提供的算法和显示不与任何特定计算机、虚拟系统或者其它设备固有相关。各种通用系统也可以与基于在此的示教一起使用。根据上面的描述，构造这类系统所要求的结构是显而易见的。此外，本发明也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本发明的内容，并且上面对特定语言所做的描述是为了披露本发明的最佳实施方式。The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with teaching based on this. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not directed to any particular programming language. It is to be understood that various programming languages may be used to implement the inventions described herein, and that the descriptions of specific languages above are intended to disclose the best mode for carrying out the invention.
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be construed as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the invention within and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的多媒体视频的编辑方法及装置中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all of the components in the method and apparatus for editing multimedia video according to the embodiments of the present invention . The present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。It should be noted that the above-described embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.
本发明的实施例公开了：Embodiments of the present invention disclose:
A1、一种多媒体视频的编辑方法，包括：A1. A method for editing a multimedia video, comprising:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
A2、根据A1所述的方法，所述对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理包括：A2. The method according to A1, wherein the rendering processing on the video data and the audio track processing on the audio data include:
接收用户输入的处理指令，所述处理指令中携带有效果标识；receiving a processing instruction input by the user, where the processing instruction carries an effect identifier;
根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。The video data is rendered according to the video effect identification in the effect identification, and the audio data is processed according to the audio effect identification in the effect identification.
A3、根据A2所述的方法，所述根据所述效果标识中的视频效果标识渲染所述视频数据包括：A3. The method according to A2, wherein rendering the video data according to the video effect identifier in the effect identifier includes:
提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；Extracting the image data of each frame in the video data, and performing filter processing on the image data;
根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。The target image in the filter-processed image data is identified according to the video effect identifier, and the target image is synthesized and rendered.
A4、根据A3所述的方法，所述根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染包括：A4. The method according to A3, wherein identifying the target image in the filtered image data according to the video effect identifier, and performing composite rendering on the target image includes:
若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。If it is identified that the video effect is identified as a composite stereoscopic image, the target image is segmented, and the target image, the segmented target image and the rendered image are colored and synthesized according to a preset coloring rule, and the preset coloring is performed. The rule is used to reflect the position display relationship among the target image, the segmented target image, and the rendered image.
A5、根据A2所述的方法，所述根据所述效果标识中的音频效果标识处理所述音频数据包括：A5. The method according to A2, wherein the processing of the audio data according to the audio effect identifier in the effect identifier includes:
按照预设时间间隔采集所述音频数据中的离散音轨数据；Collect discrete audio track data in the audio data according to preset time intervals;
根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。The discrete audio track data and the preset audio track are effectively superimposed according to the audio effect identifier.
A6、根据A1所述的方法，所述解码所述多媒体文件中的视频数据及音频数据包括：A6. The method according to A1, wherein the decoding of video data and audio data in the multimedia file includes:
按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。The video data and audio data in the multimedia file are decoded according to the video track and the audio track, respectively.
A7、根据A1所述的方法，所述对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理之后，所述方法还包括：A7. The method according to A1, after the rendering processing is performed on the video data and the audio track processing is performed on the audio data, the method further includes:
当接收到实时预览请求时，展示所述视频数据及所述音频数据。The video data and the audio data are displayed when a real-time preview request is received.
A8、根据A1所述的方法，所述方法还包括：A8. The method according to A1, further comprising:
接收速度调整指令，根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。A speed adjustment instruction is received, and the playback speed of video data and audio data in the multimedia video is adjusted according to the speed information carried in the speed adjustment instruction.
B9、一种多媒体视频的编辑装置，包括：B9, a multimedia video editing device, comprising:
获取单元，用于获取多媒体文件；an acquisition unit for acquiring multimedia files;
解码单元，用于解码所述多媒体文件中的视频数据及音频数据；a decoding unit for decoding video data and audio data in the multimedia file;
处理单元，用于对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；a processing unit, configured to perform rendering processing on the video data and track processing on the audio data;
编码单元，用于将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The encoding unit is used for encoding the processed video data and the processed audio data to obtain a multimedia video.
B10、根据B9所述的装置，所述处理单元包括：B10. The apparatus according to B9, wherein the processing unit comprises:
接收模块，用于接收用户输入的处理指令，所述处理指令中携带有效果标识；a receiving module, configured to receive a processing instruction input by a user, wherein the processing instruction carries an effect identifier;
处理模块，用于根据所述效果标识中的视频效果标识渲染所述视频数据，并根据所述效果标识中的音频效果标识处理所述音频数据。A processing module, configured to render the video data according to the video effect identification in the effect identification, and process the audio data according to the audio effect identification in the effect identification.
B11、根据B10所述的装置，所述处理模块包括：B11. The device according to B10, wherein the processing module includes:
提取子模块，用于提取所述视频数据中每一帧的图像数据，并对所述图像数据进行滤镜处理；Extraction submodule for extracting the image data of each frame in the video data, and performing filter processing on the image data;
合成子模块，用于根据所述视频效果标识识别滤镜处理后图像数据中的目标图像，并对所述目标图像进行合成渲染。A synthesis sub-module, configured to identify a target image in the image data after filter processing according to the video effect identifier, and perform synthesis and rendering on the target image.
B12、根据B11所述的装置，B12. The device according to B11,
所述合成子模块，具体用于若识别出所述视频效果标识为合成立体图像，则分割所述目标图像，按照预置着色规则对所述目标图像、所述分割后的目标图像以及渲染图像进行着色合成，所述预置着色规则用于反应所述目标图像、所述分割后的目标图像、所述渲染图像之间的位置显示关系。The synthesizing sub-module is specifically configured to segment the target image if it is identified that the video effect is identified as a synthesizing stereoscopic image, and perform the segmentation of the target image, the segmented target image and the rendered image according to a preset coloring rule. Perform coloring synthesis, and the preset coloring rule is used to reflect the position display relationship among the target image, the segmented target image, and the rendered image.
B13、根据B10所述的装置，所述处理模块还包括：B13. The device according to B10, the processing module further comprises:
采集子模块，用于按照预设时间间隔采集所述音频数据中的离散音轨数据；a collection sub-module for collecting discrete audio track data in the audio data according to preset time intervals;
叠加子模块，用于根据所述音频效果标识将所述离散音轨数据与预设音轨进行有效叠加。A superimposition sub-module, configured to effectively superimpose the discrete audio track data and the preset audio track according to the audio effect identifier.
B14、根据B9所述的装置，B14. The device according to B9,
所述解码单元，具体用于按照视频轨迹与音频轨迹分别解码所述多媒体文件中的视频数据及音频数据。The decoding unit is specifically configured to decode the video data and the audio data in the multimedia file according to the video track and the audio track respectively.
B15、根据B9所述的装置，所述装置还包括：B15. The device according to B9, further comprising:
展示单元，用于当接收到实时预览请求时，展示所述视频数据及所述音频数据。The display unit is configured to display the video data and the audio data when a real-time preview request is received.
B16、根据B9所述的装置，所述装置还包括：B16. The device according to B9, further comprising:
调整单元，用于接收速度调整指令，并根据所述速度调整指令中携带的速度信息调整多媒体视频中视频数据及音频数据的播放速度。The adjustment unit is configured to receive a speed adjustment instruction, and adjust the playback speed of video data and audio data in the multimedia video according to the speed information carried in the speed adjustment instruction.
C17、一种存储设备，其中存储有多条指令，所述指令适于由处理器加载并执行：C17. A storage device having stored therein a plurality of instructions adapted to be loaded and executed by a processor:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
D18、一种移动终端，包括处理器，适于实现各种指令；以及存储设备，适于存储多条指令，所述指令适于由处理器加载并执行：D18. A mobile terminal, comprising a processor adapted to implement various instructions; and a storage device adapted to store a plurality of instructions, the instructions being adapted to be loaded and executed by the processor:
获取多媒体文件；Get multimedia files;
解码所述多媒体文件中的视频数据及音频数据；decoding the video data and audio data in the multimedia file;
对所述视频数据进行渲染处理，以及对所述音频数据进行音轨处理；Performing rendering processing on the video data, and performing track processing on the audio data;
将处理后的视频数据及处理后的音频数据进行编码，得到多媒体视频。The processed video data and the processed audio data are encoded to obtain a multimedia video.
Claims (12)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710566432.2ACN107241646B (en) | 2017-07-12 | 2017-07-12 | Multimedia video editing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710566432.2ACN107241646B (en) | 2017-07-12 | 2017-07-12 | Multimedia video editing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107241646A CN107241646A (en) | 2017-10-10 |
| CN107241646Btrue CN107241646B (en) | 2020-08-14 |
Family
ID=59990913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710566432.2AActiveCN107241646B (en) | 2017-07-12 | 2017-07-12 | Multimedia video editing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107241646B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108234479A (en)* | 2017-12-29 | 2018-06-29 | 北京百度网讯科技有限公司 | For handling the method and apparatus of information |
| CN109168027B (en)* | 2018-10-25 | 2020-12-11 | 北京字节跳动网络技术有限公司 | Instant video display method and device, terminal equipment and storage medium |
| CN109543560A (en)* | 2018-10-31 | 2019-03-29 | 百度在线网络技术（北京）有限公司 | Dividing method, device, equipment and the computer storage medium of personage in a kind of video |
| CN109587552B (en)* | 2018-11-26 | 2021-06-15 | Oppo广东移动通信有限公司 | Video character sound effect processing method, device, mobile terminal and storage medium |
| CN111343499A (en)* | 2018-12-18 | 2020-06-26 | 北京奇虎科技有限公司 | A video synthesis method and device |
| CN111355960B (en)* | 2018-12-21 | 2021-05-04 | 北京字节跳动网络技术有限公司 | Method and device for synthesizing video file, mobile terminal and storage medium |
| CN111866404B (en)* | 2019-04-25 | 2022-04-29 | 华为技术有限公司 | A video editing method and electronic device |
| CN112533058A (en)* | 2019-09-17 | 2021-03-19 | 西安中兴新软件有限责任公司 | Video processing method, device, equipment and computer readable storage medium |
| CN111460183B (en)* | 2020-03-30 | 2024-02-13 | 北京金堤科技有限公司 | Method and device for generating multimedia file, storage medium and electronic equipment |
| CN111818385B (en)* | 2020-07-22 | 2022-08-09 | Oppo广东移动通信有限公司 | Video processing method, video processing device and terminal equipment |
| CN113315928B (en)* | 2021-05-25 | 2022-03-22 | 南京慕映影视科技有限公司 | Multimedia file making system and method |
| CN114007077B (en)* | 2021-11-17 | 2023-09-01 | 北京百度网讯科技有限公司 | Multimedia resource processing method, device, electronic device and storage medium |
| CN114979766B (en)* | 2022-05-11 | 2023-11-21 | 深圳市闪剪智能科技有限公司 | Audio and video synthesis method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102638658A (en)* | 2012-03-01 | 2012-08-15 | 盛乐信息技术（上海）有限公司 | Method and system for editing audio-video |
| CN103049908A (en)* | 2012-12-10 | 2013-04-17 | 北京百度网讯科技有限公司 | Method and device for generating stereoscopic video file |
| CN103327361A (en)* | 2012-11-22 | 2013-09-25 | 中兴通讯股份有限公司 | Method, device and system for obtaining real-time video communication playback data flow |
| CN104732593A (en)* | 2015-03-27 | 2015-06-24 | 厦门幻世网络科技有限公司 | Three-dimensional animation editing method based on mobile terminal |
| CN106373170A (en)* | 2016-08-31 | 2017-02-01 | 北京云图微动科技有限公司 | Video making method and video making device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080002942A1 (en)* | 2006-05-24 | 2008-01-03 | Peter White | Method and apparatus for creating a custom track |
- 2017
- 2017-07-12CNCN201710566432.2Apatent/CN107241646B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102638658A (en)* | 2012-03-01 | 2012-08-15 | 盛乐信息技术（上海）有限公司 | Method and system for editing audio-video |
| CN103327361A (en)* | 2012-11-22 | 2013-09-25 | 中兴通讯股份有限公司 | Method, device and system for obtaining real-time video communication playback data flow |
| CN103049908A (en)* | 2012-12-10 | 2013-04-17 | 北京百度网讯科技有限公司 | Method and device for generating stereoscopic video file |
| CN104732593A (en)* | 2015-03-27 | 2015-06-24 | 厦门幻世网络科技有限公司 | Three-dimensional animation editing method based on mobile terminal |
| CN106373170A (en)* | 2016-08-31 | 2017-02-01 | 北京云图微动科技有限公司 | Video making method and video making device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107241646A (en) | 2017-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241646B (en) | Multimedia video editing method and device | |
| US11482192B2 (en) | Automated object selection and placement for augmented reality | |
| CN108200446B (en) | On-line multimedia interaction system and method of virtual image | |
| CN110602554A (en) | Cover image determining method, device and equipment | |
| CN109327727A (en) | A live streaming processing method and streaming client in WebRTC | |
| JP7511026B2 (en) | Image data encoding method and device, display method and device, and electronic device | |
| McIlvenny | Inhabiting spatial video and audio data: Towards a scenographic turn in the analysis of social interaction | |
| CN113395569B (en) | Video generation method and device | |
| KR20160119218A (en) | Sound image playing method and device | |
| US9530453B2 (en) | Apparatus, method, and computer-readable recording medium for creating and reproducing live picture file | |
| CN104580837A (en) | Video director engine based on GPU+CPU+IO architecture and using method thereof | |
| CN104091608A (en) | Video editing method and device based on IOS equipment | |
| CN105872827A (en) | Live broadcast method and device of application interface in mobile terminal | |
| CN105763923A (en) | Video and video template editing methods and device thereof | |
| CN106604047A (en) | Multi-video-stream video direct broadcasting method and device | |
| US20180330167A1 (en) | Personalized Augmented Reality | |
| US11330307B2 (en) | Systems and methods for generating new content structures from content segments | |
| CN106792219B (en) | It is a kind of that the method and device reviewed is broadcast live | |
| CN113709521B (en) | System for automatically matching background according to video content | |
| CN114422813A (en) | VR live video splicing and displaying method, device, equipment and storage medium | |
| CN114398517A (en) | Video data acquisition method and device | |
| CN107426487A (en) | A kind of panoramic picture recorded broadcast method and system | |
| JP2007519068A (en) | Computer-based calculation method and computer system for generating a semantic description using conversion technology | |
| CN105307001A (en) | Method and device for real-time displaying release information on video program | |
| Wehmeier | Professional Amateurship and Haptic Intensity: Poetry Slam on YouTube |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |