CN115016871A - Multimedia editing method, electronic device and storage medium - Google Patents
Multimedia editing method, electronic device and storage mediumDownload PDFInfo
- Publication number
- CN115016871A CN115016871ACN202111615710.1ACN202111615710ACN115016871ACN 115016871 ACN115016871 ACN 115016871ACN 202111615710 ACN202111615710 ACN 202111615710ACN 115016871 ACN115016871 ACN 115016871A
- Authority
- CN
- China
- Prior art keywords
- editing
- application
- interface
- video
- multimedia
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Stored Programmes (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及智能终端技术领域，具体涉及一种多媒体编辑方法、电子设备及存储介质。The invention relates to the technical field of intelligent terminals, in particular to a multimedia editing method, an electronic device and a storage medium.
背景技术Background technique
随着短视频类多媒体产业的蓬勃发展，人们也越来越习惯于对所拍摄的视频或图片添加上喜欢的音乐或音效等处理成短视频，用于分享或保存欣赏等。由此，有些终端电子设备可实现音/视频编辑，来提高人们编辑短视频的操作便利性。参考图1a至图1b所示，用户在手机100的图库应用中浏览视频时，可以在图1a所示的视频浏览界面101上点击编辑按钮011跳转至图1b所示的视频编辑应用的编辑界面102，此时手机100相应地切换至运行视频编辑应用，其中，该视频编辑应用例如可以是剪映TM、抖音TM等具有短视频编辑功能的应用。用户在图1b所示界面上完成添加音频、添加变速、动画等操作后，可以点击编辑界面102右上角的导出按钮021导出完成编辑的短视频。With the vigorous development of the short video multimedia industry, people are more and more accustomed to adding their favorite music or sound effects to the captured videos or pictures and processing them into short videos for sharing or saving for enjoyment. As a result, some terminal electronic devices can implement audio/video editing to improve the operational convenience for people to edit short videos. 1a to 1b, when the user browses videos in the gallery application of the
然而，上述图1a至图1b所示的跨应用进行音/视频编辑的过程中，手机100需要由运行一个应用切换至运行另一个应用，由于应用的启动、跳转等需要时间，因此这个过程必然会产生时延。同时，用户在完成编辑之后如果想继续返回图库应用浏览其他视频或照片时，需要先退出手机100当前运行的视频编辑应用才能返回图库应用的界面，或者需重新点击手机100桌面的图库应用图标再次进入图库应用的界面，操作比较繁琐，同时手机100切换应用导致的界面跳转，也会导致用户的视觉体验不够流畅，进而导致用户体验不佳。However, in the process of performing audio/video editing across applications as shown in FIG. 1a to FIG. 1b, the
发明内容SUMMARY OF THE INVENTION
本申请实施例提供了一种多媒体编辑方法、电子设备及存储介质，能够使音/视频应用跨进程使用音/视频编辑应用的编辑能力，实现在不切换当前音/视频应用的显示界面的情况下完成音/视频编辑处理的过程，编辑处理的结果还能够在音/视频应用的界面直接展示给用户，使得用户视觉更加流畅，利于提高用户的使用体验。Embodiments of the present application provide a multimedia editing method, an electronic device, and a storage medium, which can enable an audio/video application to use the editing capability of the audio/video editing application across processes, and realize the situation without switching the display interface of the current audio/video application The process of audio/video editing processing is completed in the next step, and the results of the editing processing can also be directly displayed to the user on the interface of the audio/video application, so that the user's vision is smoother and the user experience is improved.
第一方面，本申请实施例提供了一种多媒体编辑方法，应用于安装有第一应用和第二应用的电子设备，该方法包括：电子设备显示第一应用的第一界面，第一界面包括第一多媒体数据；检测到用户对第一多媒体数据输入第一编辑操作，电子设备显示第一应用的第二界面，其中第二界面包括一个或多个编辑控件，编辑控件用于响应用户操作对第一多媒体数据进行相应的编辑处理；检测到用户为第一多媒体数据选中第一编辑控件，电子设备通过调用第二应用的多媒体编辑能力对第一多媒体数据执行第一编辑处理，得到第二多媒体数据；电子设备在第二界面中显示第二多媒体数据。In a first aspect, an embodiment of the present application provides a multimedia editing method, which is applied to an electronic device installed with a first application and a second application. The method includes: the electronic device displays a first interface of the first application, and the first interface includes: The first multimedia data; after detecting that the user inputs a first editing operation to the first multimedia data, the electronic device displays a second interface of the first application, wherein the second interface includes one or more editing controls, and the editing controls are used for Performing corresponding editing processing on the first multimedia data in response to the user operation; detecting that the user selects the first editing control for the first multimedia data, the electronic device modifies the first multimedia data by invoking the multimedia editing capability of the second application The first editing process is performed to obtain second multimedia data; the electronic device displays the second multimedia data in the second interface.
即用户可以通过电子设备上的第一应用浏览第一多媒体数据，并且可以在展示第一多媒体数据的第一界面上，选择对第一多媒体数据进行编辑。如果用户选择对第一多媒体数据进行编辑，则电子设备是显示界面可以切换至第一应用的第二界面来展示第一多媒体数据，用户可以进一步在第二界面上点击编辑控件对第一多媒体数据进行编辑操作。用户在第二界面上进行的编辑操作，可以跨进程通过电子设备所运行的第二应用的多媒体编辑能力来执行相应的编辑处理，而编辑处理的结果可以在第一应用的第二界面上呈现，即编辑处理的预览效果可以显示在第二界面上。That is, the user can browse the first multimedia data through the first application on the electronic device, and can choose to edit the first multimedia data on the first interface displaying the first multimedia data. If the user chooses to edit the first multimedia data, the display interface of the electronic device can be switched to the second interface of the first application to display the first multimedia data, and the user can further click the editing control on the second interface to display the first multimedia data. Editing operations are performed on the first multimedia data. The editing operation performed by the user on the second interface can perform corresponding editing processing through the multimedia editing capability of the second application running on the electronic device across processes, and the result of the editing processing can be presented on the second interface of the first application , that is, the preview effect of the editing process can be displayed on the second interface.
作为示例，上述第一应用例如可以是下文实施例中描述的图库、AOD等音/视频应用，上述第二应用例如是下文实施例中描述的花瓣剪辑TM等音/视频编辑应用。因此，上述第二应用的多媒体编辑能力，即是音/视频编辑应用对多媒体数据的编辑处理能力，例如下文实施例中图4所示步骤404中描述的编辑能力，包括分割、变速(或快慢速)、音量、动画、滤镜等。上述第一应用的第一界面例如可以是下文实施例中图2a等所示的视频浏览界面，上述第一应用的第二界面例如可以是下文实施例中图2b等所示的视频编辑界面。上述第二界面所包括的一个或多个编辑控件，例如可以是下文实施例中图2b所示的编辑功能菜单栏230中的各个编辑按钮等。用户可以点击编辑功能菜单栏230中的各个编辑按钮进行相应的音/视频编辑操作。As an example, the above-mentioned first application may be, for example, an audio/video application such as gallery and AOD described in the following embodiments, and the above-mentioned second application may be, for example, an audio/video editing application such as Petal ClipTM described in the following embodiments. Therefore, the multimedia editing capability of the above-mentioned second application is the editing processing capability of the audio/video editing application for multimedia data, such as the editing capability described in
可以理解，用户在音/视频应用的视频编辑界面(即上述第二界面)点击任意一个编辑按钮，手机100等电子设备都可以通过所运行的音/视频编辑应用提供的对应于用户所点击编辑按钮的编辑能力，对当前视频编辑界面上显示的第一多媒体数据进行编辑处理，得到第二多媒体数据。如果用户多次点击编辑按钮，进行多次编辑操作，相应的音/视频编辑应用执行完每一次对应于用户编辑操作的编辑处理，都会得到一个第二多媒体数据，多次编辑处理的结果可以叠加。也就是说，多次编辑处理得到的第二多媒体数据，都可以保留。It can be understood that, if the user clicks any edit button on the video editing interface of the audio/video application (that is, the above-mentioned second interface), the electronic devices such as the
可以理解，用户撤销所进行的编辑操作，相应的编辑处理结果也可以相应撤销。当然，也可以将用户的撤销操作理解为一种编辑操作，电子设备所运行的音/视频编辑应用执行该编辑操作即可。It can be understood that, when the user cancels the editing operation performed, the corresponding editing processing result can also be correspondingly canceled. Of course, the user's undo operation can also be understood as an editing operation, and the audio/video editing application running on the electronic device only needs to perform the editing operation.
在上述第一方面的一种可能的实现中，上述第一界面包括第一编辑按钮，并且用户对第一多媒体数据输入第一编辑操作，为作用于第一编辑按钮的操作。In a possible implementation of the first aspect, the first interface includes a first edit button, and the user inputs a first edit operation on the first multimedia data, which is an operation acting on the first edit button.
上述第一界面例如是下文实施例中图2a等所示的视频浏览界面，该视频浏览界面上显示有能够对当前浏览的视频的编辑按钮，用户对第一多媒体数据输入第一编辑操作，例如可以是下文实施例中，用户在图2a所示的视频浏览界面上点击编辑按钮211的操作。The above-mentioned first interface is, for example, the video browsing interface shown in FIG. 2 a in the following embodiments, and the video browsing interface is displayed with an edit button that can edit the currently browsed video, and the user inputs a first editing operation to the first multimedia data. For example, in the following embodiment, the user clicks the
在上述第一方面的一种可能的实现中，电子设备在第一进程上运行第一应用，电子设备在第二进程上运行第二应用；检测到用户对第一多媒体数据输入第一编辑操作，电子设备显示第一应用的第二界面，包括：响应于第一编辑操作，电子设备从第一进程向第二进程发送第一请求，第一请求用于向第二进程请求第二应用的多媒体编辑能力列表；基于第二进程返回的多媒体编辑能力列表，电子设备生成在第二界面上显示的一个或多个编辑控件；电子设备显示第二界面。In a possible implementation of the above-mentioned first aspect, the electronic device runs the first application on the first process, and the electronic device runs the second application on the second process; it is detected that the user inputs the first multimedia data into the first application. In an editing operation, the electronic device displays the second interface of the first application, including: in response to the first editing operation, the electronic device sends a first request from the first process to the second process, where the first request is used to request the second process for a second The multimedia editing capability list of the application; based on the multimedia editing capability list returned by the second process, the electronic device generates one or more editing controls displayed on the second interface; the electronic device displays the second interface.
即第一应用在电子设备系统运行的第一进程上运行，第二应用在电子设备系统运行的第二进程上运行，用户在第一应用的界面上的操作对应生成的操作请求(包括上述第一请求)，能够跨进程从第一进程发送给第二进程，由运行在第二进程上的第二应用来处理。上述第一请求例如可以是下文实施例中图4所示步骤402中描述的“编辑能力请求”，用于获取视频编辑应用的编辑能力，第二应用即为该视频编辑应用，上述多媒体编辑能力即为该视频编辑应用的编辑能力。上述第一应用例如是下文实施例中描述的图库311，则第二进程向运行第一应用的第一进程所返回的多媒体编辑能力列表，例如可以是返回给图库311的“编辑能力列表”，图库311(即第一应用)的视频编辑界面(即上述第二界面)上显示的编辑控件的类型以及所对应的编辑功能，与编辑能力列表上的各个编辑能力相对应。That is, the first application runs on the first process run by the electronic device system, the second application runs on the second process run by the electronic device system, and the user's operation on the interface of the first application corresponds to the generated operation request (including the above-mentioned No. A request), which can be sent across processes from a first process to a second process, for processing by a second application running on the second process. The above-mentioned first request may be, for example, the “editing capability request” described in
可以理解，第一进程上运行的第一应用不同时，第二进程返回的多媒体编辑能力列表也会有所不同，具体可以参考下文实施例中步骤404中的相关描述，在此不做赘述。It can be understood that when the first application running on the first process is different, the multimedia editing capability list returned by the second process will also be different. For details, please refer to the relevant description in
在上述第一方面的一种可能的实现中，上述方法还包括：多个编辑控件中的各编辑控件与多媒体编辑能力列表中的各个多媒体编辑能力一一对应。In a possible implementation of the above-mentioned first aspect, the above-mentioned method further includes: each editing control in the plurality of editing controls corresponds to each multimedia editing capability in the multimedia editing capability list one-to-one.
在上述第一方面的一种可能的实现中，检测到用户为第一多媒体数据选中第一编辑控件，包括：检测到用户在第二界面上作用于第一编辑控件的第二编辑操作。In a possible implementation of the above-mentioned first aspect, detecting that the user selects the first editing control for the first multimedia data includes: detecting a second editing operation performed by the user on the first editing control on the second interface .
第二界面例如是下文实施例中描述的视频编辑界面，编辑控件例如是下文实施例中描述的编辑功能按钮。即用户可以在视频编辑界面上，通过点击编辑功能按钮进行相应的编辑操作。上述第一编辑控件可以是第二界面上显示多个编辑控件中的任意一个，即下文实施例中描述的编辑功能菜单栏中的任意一个编辑功能按钮。可以理解，编辑功能菜单栏中的各个编辑功能按钮(即各个编辑控件)分别对应于不同的编辑能力、也对应于不同的编辑处理效果。The second interface is, for example, the video editing interface described in the following embodiments, and the editing controls are, for example, editing function buttons described in the following embodiments. That is, the user can perform the corresponding editing operation by clicking the editing function button on the video editing interface. The above-mentioned first editing control may be any one of a plurality of editing controls displayed on the second interface, that is, any editing function button in the editing function menu bar described in the following embodiments. It can be understood that each editing function button (ie, each editing control) in the editing function menu bar corresponds to different editing capabilities and also corresponding to different editing processing effects.
在上述第一方面的一种可能的实现中，电子设备通过调用第二应用的多媒体编辑能力对第一多媒体数据执行第一编辑处理，包括：响应于第二编辑操作，电子设备从第一进程向第二进程发送第一编辑指令，第一编辑指令用于指示第二进程对第一多媒体数据执行第一编辑处理；电子设备运行的第二进程响应第一编辑指令，基于第二应用的多媒体编辑能力执行第一编辑处理。In a possible implementation of the above-mentioned first aspect, the electronic device performs the first editing process on the first multimedia data by invoking the multimedia editing capability of the second application, including: in response to the second editing operation, A process sends a first editing instruction to the second process, where the first editing instruction is used to instruct the second process to perform a first editing process on the first multimedia data; the second process running on the electronic device responds to the first editing instruction, based on the first editing instruction The multimedia editing capability of the second application performs the first editing process.
即用户在第一应用的第二界面上进行的第二编辑操作，可以触发电子设备内的跨进程编辑处理。即第二编辑操作对应的第一编辑指令，从运行第一应用的第一进程，跨进程交给运行第二应用的第二进程来处理。第二进程接收到第一编辑指令后，可以通过运行在第二进程上的第二应用具有的编辑能力对第一多媒体数据进行相应的编辑处理，即上述第一编辑处理。上述第一编辑指令，例如可以是下文实施例中图4以及图7的相关描述中的编辑操作指令，该编辑操作指令与用户在视频编辑界面上点击哪一项编辑功能按钮相关联。例如用户点击了滤镜按钮，则该编辑操作指令则用于指示视频编辑应用所在的进程，对第一媒体数据进行滤镜处理。That is, the second editing operation performed by the user on the second interface of the first application can trigger the cross-process editing processing in the electronic device. That is, the first editing instruction corresponding to the second editing operation is handed over from the first process running the first application to the second process running the second application across the processes for processing. After receiving the first editing instruction, the second process can perform corresponding editing processing on the first multimedia data through the editing capability of the second application running on the second process, that is, the above-mentioned first editing processing. The above-mentioned first editing instruction may be, for example, an editing operation instruction in the related description of FIG. 4 and FIG. 7 in the following embodiments, and the editing operation instruction is associated with which editing function button the user clicks on the video editing interface. For example, if the user clicks the filter button, the editing operation instruction is used to instruct the process of the video editing application to perform filter processing on the first media data.
在上述第一方面的一种可能的实现中，电子设备通过调用第二应用的多媒体编辑能力对第一多媒体数据执行第一编辑处理，还包括：电子设备运行的第二进程向第一进程反馈第一编辑处理的执行结果。In a possible implementation of the above-mentioned first aspect, the electronic device performs the first editing process on the first multimedia data by invoking the multimedia editing capability of the second application, and further includes: the second process running by the electronic device reports to the first The process feeds back the execution result of the first editing process.
即电子设备所运行的第二进程完成对第一多媒体数据的编辑处理后，可以通知第一进程已完成该项编辑处理，例如，第二进程可以向第二进程发送完成编辑处理的返回值，参考下文实施例中的步骤的408或步骤908中相关描述，在此不做赘述。That is, after the second process running on the electronic device completes the editing process of the first multimedia data, it can notify the first process that the editing process has been completed. For example, the second process can send a return of the completion of the editing process to the second process. For the value, refer to the relevant description in
在上述第一方面的一种可能的实现中，电子设备在第二界面中显示第二多媒体数据，包括：电子设备在第二界面上播放第二多媒体数据以预览画面效果，和/或电子设备播放第二多媒体数据以预览声音效果。In a possible implementation of the above-mentioned first aspect, the electronic device displaying the second multimedia data on the second interface includes: the electronic device playing the second multimedia data on the second interface to preview the screen effect, and /or the electronic device plays the second multimedia data to preview the sound effect.
即完成编辑处理的第一多媒体数据(例如视频)，可以在电子设备当前显示的第一应用的第二界面上进行预览。可以理解，对第一多媒体数据的编辑处理可以包括画面的处理。和/或声音的处理，画面的处理例如包括下文实施例中描述的滤镜、贴纸等，声音的处理包括下文实施例中描述的混音特效以及添加音频、音效等。手机100等电子设备上显示完成编辑处理的视频预览效果时，即可以播放带有滤镜、贴纸等画面，也可以同步播放用户操作添加的音效、混音特效等。That is, the edited first multimedia data (eg, video) can be previewed on the second interface of the first application currently displayed on the electronic device. It can be understood that the editing processing of the first multimedia data may include the processing of pictures. And/or sound processing, picture processing includes, for example, filters, stickers, etc. described in the following embodiments, and sound processing includes sound mixing special effects and adding audio, sound effects, etc. described in the following embodiments. When the edited video preview effect is displayed on the electronic device such as the
在上述第一方面的一种可能的实现中，第二界面包括保存控件，并且在电子设备在第二界面中显示第二多媒体数据之后，方法还包括：电子设备检测到作用于保存控件上的保存操作；电子设备响应于保存操作，从第一进程向第二进程发送保存指令；电子设备运行的第二进程响应于保存指令，保存第二多媒体数据至第一目录下；其中第一目录为第一应用、第二应用均具有访问权限的存储空间。In a possible implementation of the above first aspect, the second interface includes a save control, and after the electronic device displays the second multimedia data in the second interface, the method further includes: the electronic device detects that the save control acts on the electronic device The electronic device sends a save instruction from the first process to the second process in response to the save operation; the second process running on the electronic device saves the second multimedia data in the first directory in response to the save instruction; wherein The first directory is a storage space where both the first application and the second application have access rights.
即用户在第一应用的第二界面上点击保存控件，则相应的保存指令可以从运行第一应用的第一进程发送至第二进程。第二进程基于接收到的保存指令，可以将完成编辑处理的第一多媒体数据(例如视频)等已编码缓存的视频流文件，进行合并、打包，保存为视频文件。该视频文件例如可以是下文实施例中步骤836中描述的短视频文件。That is, the user clicks the save control on the second interface of the first application, and a corresponding save instruction can be sent from the first process running the first application to the second process. Based on the received save instruction, the second process can combine and package the encoded and cached video stream files such as the first multimedia data (eg video) after editing, and save them as video files. The video file may be, for example, the short video file described in step 836 in the following embodiments.
在上述第一方面的一种可能的实现中，第一进程与第二进程之间通过跨进程通信技术实现交互，并且跨进程通信技术包括下列中的任一项：管道通信；基于命名管道的跨进程通信；基于内存映射的跨进程通信；基于消息队列的跨进程通信；基于共享内存的跨进程通信；基于信号量或信号的跨进程通信；基于套接字的跨进程通信。In a possible implementation of the above-mentioned first aspect, the interaction between the first process and the second process is implemented through a cross-process communication technology, and the cross-process communication technology includes any one of the following: pipe communication; Cross-process communication; memory-map-based cross-process communication; message queue-based cross-process communication; shared memory-based cross-process communication; semaphore or signal-based cross-process communication; socket-based cross-process communication.
在上述第一方面的一种可能的实现中，多个编辑控件中的各编辑控件，包括下列中的任一项：分割；变速；音量；动画；滤镜；单帧导出；裁剪；蒙版；色度抠图；镜像；倒放；定格；画中画；特效；贴纸；添加音乐/音效；添加字幕。In a possible implementation of the above-mentioned first aspect, each editing control in the plurality of editing controls includes any one of the following: segmentation; variable speed; volume; animation; filter; single-frame export; cropping; mask ; Chroma matting; Mirror; Reverse; Freeze; PIP; Effects; Stickers; Add music/sound effects; Add subtitles.
在上述第一方面的一种可能的实现中，编辑控件的形式，包括下列中的任一项：按钮、进度条、日期/时间控件、上传控件、列表框、树形控件、页面视图、输入框、多格式文本框、Tab控件、下拉选框。In a possible implementation of the above-mentioned first aspect, the form of the edit control includes any one of the following: a button, a progress bar, a date/time control, an upload control, a list box, a tree control, a page view, an input Box, rich text box, Tab control, drop-down selection.
第二方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器；一个或多个存储器；一个或多个存储器存储有一个或多个程序，当一个或者多个程序被一个或多个处理器执行时，使得电子设备上述多媒体编辑方法。In a second aspect, an embodiment of the present application provides an electronic device, including: one or more processors; one or more memories; one or more memories are stored with one or more programs, when the one or more programs are When executed by one or more processors, the electronic device makes the above multimedia editing method.
第三方面，本申请实施例提供了一种计算机可读存储介质，存储介质上存储有指令，指令在计算机上执行时使计算机执行上述多媒体编辑方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where instructions are stored on the storage medium, and when the instructions are executed on a computer, the computer executes the above multimedia editing method.
第四方面，本申请实施例提供了一种计算机程序产品，包括计算机程序/指令，计算机程序/指令被电子设备执行时实现上述多媒体编辑方法。In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program/instruction, which implements the above multimedia editing method when the computer program/instruction is executed by an electronic device.
附图说明Description of drawings
图1a至图1b所示为一些用户界面(User Interface，UI)示意图。1a to 1b are schematic diagrams of some user interfaces (User Interface, UI).
图2a至图2d所示为本申请实施例提供的一些UI界面示意图。FIG. 2a to FIG. 2d are schematic diagrams of some UI interfaces provided by the embodiments of the present application.
图3所示为本申请实施例提供的一种手机100的软件系统架构示意框图。FIG. 3 shows a schematic block diagram of a software system architecture of a
图4所示为本申请实施例1提供的一种多媒体编辑方法的实施流程示意图。FIG. 4 is a schematic diagram illustrating an implementation flow of a multimedia editing method provided in Embodiment 1 of the present application.
图5a至图5g所示为本申请实施例1提供的一些UI界面示意图。5a to 5g are schematic diagrams of some UI interfaces provided in Embodiment 1 of the present application.
图6a至图6d所示为本申请实施例1提供的另一些UI界面示意图。6a to 6d are schematic diagrams of other UI interfaces provided in Embodiment 1 of the present application.
图7所示为本申请实施例1提供的应用进程之间的交互时序示意图。FIG. 7 is a schematic diagram of an interaction sequence between application processes according to Embodiment 1 of the present application.
图8a至图8c所示为本申请实施例1提供的应用进程之间的具体交互流程示意图。8a to 8c are schematic diagrams showing a specific interaction flow between application processes according to Embodiment 1 of the present application.
图9所示为本申请实施例2提供的一种多媒体编辑方法的实施流程示意图。FIG. 9 is a schematic diagram illustrating an implementation flow of a multimedia editing method provided in Embodiment 2 of the present application.
图10a至图10g所示为本申请实施例2提供的一些UI界面示意图。10a to 10g are schematic diagrams of some UI interfaces provided in Embodiment 2 of the present application.
图11所示为本申请实施例提供的一种编辑能力服务的组成结构示意图。FIG. 11 is a schematic diagram showing the composition and structure of an editing capability service provided by an embodiment of the present application.
图12所示为本申请实施例2提供的一种手机100的硬件结构示意图。FIG. 12 is a schematic diagram of a hardware structure of a
具体实施方式Detailed ways
在具体介绍本申请实施例所提供的多媒体编辑方法的实现过程之前，下面先就本申请实施例涉及的部分计算机技术术语进行简要解释，以便更好地理解本申请方案。Before specifically introducing the implementation process of the multimedia editing method provided by the embodiments of the present application, some computer technical terms involved in the embodiments of the present application are briefly explained below, so as to better understand the solutions of the present application.
进程，是资源(CPU、内存等)分配的基本单位，应用程序运行时系统就会创建一个进程，并为它分配资源，然后把该进程放入进程就绪队列，进程调度器选中它的时候就会为它分配CPU时间，应用程序开始真正运行。其中，进程调度器用于调度和管理进程的运行，进程调度器的一个重要目标是有效地分配CPU时间，并且在面对一些互相冲突的进程任务时，既要为关键实时任务保障最小化响应时间、又要最大限度地提高CPU的总体利用率，以提供较好的用户体验。Process is the basic unit of resource (CPU, memory, etc.) allocation. When the application is running, the system will create a process, allocate resources to it, and then put the process into the process ready queue. When the process scheduler selects it, it will CPU time is allocated to it and the application starts to actually run. Among them, the process scheduler is used to schedule and manage the running of the process. An important goal of the process scheduler is to effectively allocate CPU time, and in the face of some conflicting process tasks, it is necessary to minimize the response time for critical real-time tasks. , and to maximize the overall utilization of the CPU to provide a better user experience.
会话(session)，在计算机专业术语里，是指一个终端用户与交互系统进行通信的时间间隔。通常指从注册系统到注销系统之间所经过的时间，以及系统处理业务的过程中所需的一些操作空间。示例性的，可将session翻译成会话。在本申请实施例中，session是指一个进程与另一个进程进行通信的编辑会话，因此，在本申请实施例中可以把一个进程与另一个进程之间的一系列交互动作(例如编辑处理过程中的交互)称为一个session。本申请实施例中，一个session所包括的操作空间，例如可以是两个进程之间创建的编辑会话所包括的一个进程(例如视频编辑应用进程)为另一个进程(例如图库等浏览应用进程)创建的时间轴(Timelines)等编辑环境，该操作空间可用于进行音/视频编辑处理。Session, in computer terminology, refers to the time interval during which an end user communicates with an interactive system. Usually refers to the elapsed time from registering the system to deregistering the system, as well as some operating space required in the process of the system processing business. Exemplarily, a session can be translated into a session. In this embodiment of the present application, a session refers to an editing session in which one process communicates with another process. Therefore, in this embodiment of the present application, a series of interactive actions between one process and another process (for example, an editing process interaction) is called a session. In this embodiment of the present application, the operation space included in a session may be, for example, one process (such as a video editing application process) included in an editing session created between two processes, and another process (such as a browsing application process such as a gallery) Created an editing environment such as Timelines, this operation space can be used for audio/video editing processing.
为了解决上述跨应用进行音/视频编辑切换应用的界面所导致的用户体验不佳的问题，本申请实施例提供了一种多媒体编辑方法。在该方法中，在如图库应用或相册应用、录音应用等的音/视频应用中，可以向用户显示对浏览的音/视频文件进行编辑的编辑界面。然后通过跨进程通信技术，将用户在编辑界面中的编辑操作，发送给实际进行音/视频编辑的音/视频编辑应用进行各种编辑操作。音/视频编辑应用执行完编辑操作相应的编辑处理后，会将完成编辑处理的返回值发送给音/视频应用，以在音/视频应用中向用户展示编辑后的音/视频。音/视频编辑应用例如可以是剪映TM、抖音TM、花瓣剪辑TM等，编辑后的音/视频例如可以是剪辑得到短视频等，用户则可以在图库应用的浏览界面上查看剪辑得到的短视频。In order to solve the problem of poor user experience caused by the above-mentioned cross-application audio/video editing and switching application interface, an embodiment of the present application provides a multimedia editing method. In this method, in an audio/video application such as a gallery application, a photo album application, a recording application, etc., an editing interface for editing the browsed audio/video files can be displayed to the user. Then, through the cross-process communication technology, the user's editing operations in the editing interface are sent to the audio/video editing application that actually performs audio/video editing for various editing operations. After the audio/video editing application has performed the editing process corresponding to the editing operation, it will send the return value of the completed editing process to the audio/video application, so as to display the edited audio/video to the user in the audio/video application. The audio/video editing application can be, for example, Cut MovieTM , DouyinTM , Petal ClipTM , etc. The edited audio/video can be, for example, a short video obtained by editing, and the user can view the editing obtained on the browsing interface of the gallery application. short video.
基于本申请实施例所提供的多媒体编辑方案，能够使音/视频应用跨进程使用音/视频编辑应用的编辑能力，使得视频编辑应用的编辑能力能够在最大范围内得以复用，也大大减少了系统ROM占用；编辑处理的结果还能够在音/视频应用的界面直接展示给用户，也实现了在不切换当前音/视频应用的显示界面的情况下完成音/视频编辑处理的过程，使得用户视觉更加流畅，利于提高用户使用体验。Based on the multimedia editing solution provided by the embodiments of the present application, the audio/video application can use the editing capability of the audio/video editing application across processes, so that the editing capability of the video editing application can be reused to the maximum extent, and greatly reduces the The system ROM is occupied; the results of the editing processing can also be directly displayed to the user on the interface of the audio/video application, which also realizes the process of completing the audio/video editing processing without switching the display interface of the current audio/video application, so that the user can The vision is smoother, which is conducive to improving the user experience.
作为示例，图2a至图2d根据本申请实施例示出了一些UI示意图。其中，图2a至图2d所示的手机100上应用了本申请实施例所提供的多媒体编辑方案。As an example, FIGS. 2 a to 2 d show some UI schematic diagrams according to embodiments of the present application. The multimedia editing solution provided by the embodiments of the present application is applied to the
图2a所示为用户操作手机100打开图库应用后点击某个视频文件对应显示的视频浏览界面210，其中，图2a所示的视频浏览界面210与上述图1a所示的视频浏览界面101相同。Fig. 2a shows a
用户可以在图2a所示的视频浏览界面210上点击编辑按钮211，此时手机100保持运行图库应用，并显示图2b所示的图库应用的视频编辑界面220。与图2a所示的视频浏览界面210相比，图2b所示的视频编辑界面220中的编辑按钮211所在的菜单栏上方显示用于进行音/视频编辑的编辑功能菜单栏230。用户可以点击编辑功能菜单栏230中的各个编辑按钮进行相应的音/视频编辑操作，另外图2b所示的视频编辑界面220右上角还可以显示保存按钮240。The user can click the
参考图2b所示的操作①，用户从手机100屏幕上进行滑动返回操作，以退出图2b所示的视频编辑界面220，则手机100屏幕可以隐去视频编辑界面220上的编辑功能菜单栏230以及各个编辑按钮、和保存按钮240，显示图2a所示的视频浏览界面210。Referring to operation ① shown in FIG. 2b, the user performs a sliding return operation from the screen of the
其中，“隐去”顾名思义就是指原本显示的编辑功能菜单栏230、保存按钮240等不再显示在界面上。“隐去”也可称为退出或返回等。在图2b所示的操作①下，手机100屏幕所显示的界面由图2b变化至图2a的过程仅涉及部分功能按钮隐去或者说不显示的变化，因此从用户视角来看，手机100屏幕所显示的界面变化过程相比于在两个应用的界面之间跳转的变化过程更加流畅，利于提高用户的视觉体验。Wherein, "hidden", as the name implies, means that the originally displayed editing
参考图2b所示的操作②，用户在视频编辑界面220上点击编辑功能菜单栏230中的一个或多个编辑按钮进行编辑操作后，例如，用户点击编辑功能菜单栏230中的滤镜按钮231进行切换滤镜效果的编辑操作后，手机100屏幕可以显示图2c所示的视频编辑界面220中的效果预览250；用户进一步点击视频编辑界面220右上角的保存按钮240后，手机100屏幕则可以显示图2d所示的视频浏览界面260。Referring to operation ② shown in FIG. 2b, after the user clicks one or more editing buttons in the editing
如图2d所示，手机100屏幕显示切换滤镜效果后的视频213，切换滤镜效果之前的视频212显示在缩略框270内。可以理解，在另一些实施例中，图2d所示的视频浏览界面260也可以不包括缩略框270或者不包括缩略框270内的视频212、视频213对应的控件等，在此不做限制。用户在图2d所示的视频浏览界面260上可以左右滑动查看视频212或者视频213之后的视频或图像，在此不做赘述。As shown in FIG. 2d , the screen of the
基于上述图2a至图2d所示的界面变化，可以理解，应用了本申请的音/视频编辑方案的手机100上，既可以在图库应用的界面(参考图2a所示视频浏览界面210)上浏览音/视频文件，也可以在图库应用的界面(图2b所示的视频编辑界面220)上对正在浏览的音/视频文件进行编辑。而手机100的系统内可以通过跨进程通信技术，将用户在图库应用的视频编辑界面220上进行的编辑操作，以相应的操作指令发送给实际进行音/视频编辑处理的视频编辑应用所在进程处理。编辑处理得到的音/视频文件例如可以是上述图2d所示的视频213。Based on the interface changes shown in FIGS. 2 a to 2 d , it can be understood that on the
可见，用户进行编辑操作的过程中，手机100始终显示图库应用的界面，各项编辑操作的处理结果也可以实时展示在图库应用的编辑界面上供用户参考。如此，用户浏览音/视频时进行音/视频编辑操作的视觉体验会更加流畅，利于提高用户使用体验。It can be seen that during the editing operation by the user, the
可以理解，本申请实施例所提供的多媒体编辑方法所适用的电子设备，可以包括但不限于手机、平板电脑、桌面型、膝上型、手持计算机、上网本，以及增强现实(augmentedreality，AR)\虚拟现实(virtual reality，VR)设备、智能电视、智能手表、监控设备等具有一个或多个处理器电子设备，在此不做限制。下面继续以电子设备是手机100为例，具体介绍本申请实施例所提供的多媒体编辑方法的实现过程。It can be understood that the electronic devices to which the multimedia editing method provided in the embodiments of the present application is applicable may include but are not limited to mobile phones, tablet computers, desktops, laptops, handheld computers, netbooks, and augmented reality (AR) \ Virtual reality (virtual reality, VR) devices, smart TVs, smart watches, monitoring devices, etc. have one or more processor electronic devices, which are not limited herein. Continuing to take the electronic device being the
可以理解，本申请实施例所描述的音/视频应用为能够展示音/视频文件的应用程序，例如上文示例的图库应用或相册应用、录音应用，以及下文示例的息屏显示应用等，在本申请实施例中这些音/视频应用还可以满足用户对所展示的音/视频文件的编辑需求。It can be understood that the audio/video applications described in the embodiments of the present application are applications that can display audio/video files, such as the gallery application or photo album application, recording application in the example above, and the screen display application in the example below. These audio/video applications in the embodiments of the present application can also meet the user's editing requirements for the displayed audio/video files.
本申请实施例所描述的音/视频编辑应用为具有编辑能力的应用程序，例如上文示例的剪映TM、抖音TM、花瓣剪辑TM等，在本申请实施例中音/视频编辑应用可以为上述音/视频应用提供编辑能力服务，该编辑能力服务可以在安装相应的音/视频编辑应用时部署在手机100等电子设备的系统中。The audio/video editing applications described in the embodiments of the present application are applications with editing capabilities, such as the clippingTM , DouyinTM , and Petal EditingTM exemplified above. In the embodiments of the present application, the audio/video editing applications can be An editing capability service is provided for the above audio/video application, and the editing capability service can be deployed in a system of an electronic device such as a
本申请实施例所提供的多媒体编辑方法，所编辑的对象为多媒体数据，该多媒体数据既包括音频数据、视频数据，也包括图片图像数据等，其中音频数据可以是音乐、录音、音效以及音频片段等，视频数据可以是相机录制的视频、下载的视频文件、短视频、或者截图的视频片段等，而图片图像数据可以是相机拍摄的照片、下载的图片、编辑处理得到的图像等，在此不做限制。In the multimedia editing method provided by the embodiment of the present application, the edited object is multimedia data, and the multimedia data includes both audio data, video data, and picture image data, etc., wherein the audio data can be music, recordings, sound effects, and audio clips etc., the video data can be the video recorded by the camera, the downloaded video file, the short video, or the video clip of the screenshot, etc., and the picture image data can be the photos taken by the camera, the downloaded pictures, the images obtained by editing, etc., here No restrictions.
图3根据本申请实施例示出了一种手机100的软件系统架构示意框图。FIG. 3 shows a schematic block diagram of a software system architecture of a
手机100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明手机100的软件结构。The software system of the
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层310，应用程序框架层320，安卓运行时(Android runtime)330和系统库340，以及内核层350。The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the
如图3所示，应用程序层310可以包括一系列应用程序包，既包括系统应用也包括第三方应用。这些应用程序包可以包括图库311，息屏显示(Always On Display，AOD)312，相机313，视频编辑314，应用程序包还可以包括地图，WLAN，导航，蓝牙，音乐，日历，短信息等应用程序，在此不做赘述。As shown in FIG. 3, the
应用程序框架层320为应用程序层310的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层320包括一些预先定义的函数。应用程序框架层320可以包括编辑能力服务321，该编辑能力服务321是供应用程序层310的视频编辑314调用以提供多媒体编辑能力的服务；应用程序框架层320还可以包括窗口管理器，内容提供器，电话管理器，资源管理器，通知管理器，视图系统等。The application framework layer 320 provides an application programming interface (API) and a programming framework for the applications of the
其中，窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。该数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。电话管理器用于提供手机100的通信功能。例如通话状态的管理(包括接通，挂断等)。资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。Among them, the window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc. Content providers are used to store and retrieve data and make these data accessible to applications. This data can include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc. The phone manager is used to provide the communication function of the
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.
安卓运行时(Android Runtime)330包括核心库和虚拟机。安卓运行时330负责安卓系统的调度和管理。其中，核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。The
应用程序层310和应用程序框架层320运行在虚拟机中。虚拟机将应用程序层310和应用程序框架层320的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。The
系统库340可以包括多个功能模块。例如：表面管理器(surface manager)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)，媒体库(Media Libraries)等。其中，表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。2D图形引擎是2D绘图的绘图引擎。The system library 340 may include multiple functional modules. For example: surface manager (surface manager), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), media library (Media Libraries) and so on. Among them, the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. 2D graphics engine is a drawing engine for 2D drawing.
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音/视频编码格式，例如:MPEG4，H.264，MP3，MP4，AVI，AAC，AVC，AMR，JPG，PNG等。The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support multiple audio/video encoding formats, such as: MPEG4, H.264, MP3, MP4, AVI, AAC, AVC, AMR, JPG, PNG, etc.
内核层350是硬件和软件之间的层。内核层350至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。Kernel layer 350 is the layer between hardware and software. The kernel layer 350 includes at least a display driver, a camera driver, an audio driver, and a sensor driver.
基于上述图3所示的手机100的软件系统框架，下面结合具体实施例详细描述本申请实施例所提供的多媒体编辑方案的具体实现过程。Based on the software system framework of the
可以理解，本申请实施例所应用的跨进程通信方式可以包括但不限于：管道(pipe)，命名管道(FIFO)，内存映射(mapped memeory)，消息队列(message queue)，共享内存(shared memory)，信号量(semaphore)，信号(signal)，套接字(Socket)。在安卓TM系统中，Android接口定义语言(Android Interface Definition Language，AIDL)可以利用上述跨进程通信方式实现手机100系统内的跨进程通信能力，在此不做限制。It can be understood that the cross-process communication methods applied in the embodiments of the present application may include but are not limited to: pipes (pipe), named pipes (FIFO), memory mapping (mapped memeory), message queue (message queue), shared memory (shared memory) ), semaphore, signal, socket. In the AndroidTM system, the Android Interface Definition Language (AIDL) can utilize the above-mentioned cross-process communication method to realize the cross-process communication capability in the
下面先结合实施例1，介绍用户操作手机100打开图库311对待编辑视频(即目标对象)进行编辑操作及保存的过程中，手机100响应于用户操作实施本申请的多媒体编辑方法的具体过程。In the following, in conjunction with Embodiment 1, in the process of the user operating the
实施例1Example 1
本申请实施例以音/视频应用是图库311为例，结合图库311与视频编辑314之间的交互过程，详细介绍本申请的多媒体编辑方法的具体实施过程。The embodiment of the present application takes the audio/video application as the gallery 311 as an example, and combines the interaction process between the gallery 311 and the video editing 314 to introduce the specific implementation process of the multimedia editing method of the present application in detail.
图4根据本申请实施例示出了一种多媒体编辑方法的实施流程示意图。FIG. 4 shows a schematic diagram of an implementation flow of a multimedia editing method according to an embodiment of the present application.
如图4所示，在手机100的软件系统中，可以通过图库311与视频编辑314之间的交互实施本申请实施例所提供的多媒体编辑方法。As shown in FIG. 4 , in the software system of the
如图4所示，该流程包括以下步骤：As shown in Figure 4, the process includes the following steps:
401：图库311检测到用户启动对目标对象编辑的操作。401: The gallery 311 detects that the user initiates an operation to edit the target object.
示例性地，用户操作手机100打开图库311后，可以选择图像或视频进行浏览。如果用户想要对正在浏览的图像或视频(即目标对象)进行编辑操作，则可以在手机100所显示的相应浏览界面上操作点击编辑按钮等。此时，手机100所运行的图库311则会检测到用户在相应浏览界面上对目标对象的编辑操作。Exemplarily, after the user operates the
可以理解，在执行本步骤401之前，手机100检测到启动运行图库311的操作时，系统会创建一个图库应用进程并为所创建的图库应用进程分配资源。进程调度器选中图库应用进程时为图库应用进程分配CPU时间，图库311开始运行。在实施本申请实施例所提供的多媒体编辑方法的过程中，具体图库应用进程与视频编辑应用进程之间的交互将在下文详细描述，在此不做赘述。It can be understood that, before performing this
图5a至图5g根据本申请实施例示出了一些UI界面示意图。5a to 5g show some schematic diagrams of UI interfaces according to embodiments of the present application.
如图5a所示，用户在手机100的桌面510上点击图库应用图标511以运行图库311，手机100则运行图库311并显示图5b所示的图库界面520。As shown in FIG. 5a, the user clicks the
如图5b所示，用户在图库界面520中点击视频类型521并选择想要浏览的视频文件，手机100则可以显示图5c所示的视频浏览界面530。As shown in FIG. 5b, the user clicks the
如图5c所示，图5c所示的视频浏览界面530与上述图2a所示的视频浏览界面210相同，用户可以在视频浏览界面530上点击编辑按钮531，使手机100所运行的图库311进入对图5c所示的目标视频532进行编辑处理的准备界面，该准备界面例如是下述步骤405所示例的图6a所示界面，在此不做赘述。As shown in Fig. 5c, the
在另一些实施例中，用户操作手机100打开图库中视频所显示的视频浏览界面也可以是其他样式，例如图5d所示的视频浏览界面540、图5e所示的视频浏览界面550以及图5f所示的视频浏览界面560等，用户可以在图5d至图5f所示界面上点击编辑按钮，启动对当前浏览的视频(即目标对象)编辑的操作，在此不做限制。In other embodiments, the video browsing interface displayed by the user operating the
在另一些实施例中，用户也可以操作手机100打开其他与图库311功能类似的应用程序，浏览图像或视频，例如图5g所示的视频浏览界面570是在另一种可以浏览图像、视频以及对视频等进行编辑的应用程序中打开想要浏览的视频对应显示的界面。用户可以点击图5g所示的风格按钮571，调出相应编辑功能的按钮来编辑视频风格等，在此不做限制。In other embodiments, the user can also operate the
402：图库311发送编辑启动请求至视频编辑314。其中，该编辑启动请求至少包括编辑会话请求、编辑能力请求以及目标对象的文件路径，编辑会话请求用于请求在图库应用进程与视频编辑应用进程之间创建编辑会话(session)，编辑能力请求用于获取视频编辑应用的编辑能力。402 : The gallery 311 sends an editing start request to the video editing 314 . The editing start request includes at least an editing session request, an editing capability request, and a file path of a target object, and the editing session request is used to request to create an editing session (session) between the gallery application process and the video editing application process, and the editing capability request uses To gain editing capabilities of video editing applications.
示例性地，手机100所运行的图库311响应于检测到的启动对目标对象编辑的操作后，可以生成编辑启动请求的数据包，该数据包可以包括上述编辑会话请求的请求数据、编辑能力请求的请求数据以及目标对象的文件路径信息。该目标对象的文件路径信息，例如是当前正在浏览的视频的文件路径信息。Exemplarily, after the image gallery 311 running on the
可以理解，在图库311响应于用户启动对目标对象编辑的操作、生成编辑启动请求的之后，手机100的系统可以先基于图库311的编辑启动请求，创建视频编辑314的应用进程启动运行视频编辑314。然后，上述图库311可将生成的编辑启动请求的数据包，通过跨进程通信技术发送至视频编辑314所在进程进行处理，具体将在下文详细描述，在此不再赘述。It can be understood that, after the gallery 311 generates an editing start request in response to the user initiating the editing operation on the target object, the system of the
403：视频编辑314基于所接收的编辑启动请求，初始化编辑操作环境，并向就绪的编辑操作环境中添加目标对象。403: The video editor 314 initializes the editing operation environment based on the received editing start request, and adds the target object to the ready editing operation environment.
示例性地，视频编辑314的应用进程运行后可以对图库311发来的编辑启动请求的数据包进行解析，获取数据包中的编辑会话请求的请求数据、编辑能力请求的请求数据、以及目标对象的文件路径信息等。Exemplarily, after the application process of the video editing 314 runs, it can parse the data packet of the editing start request sent by the gallery 311, and obtain the request data of the editing session request, the request data of the editing capability request, and the target object in the data packet. file path information, etc.
如上所述，编辑会话请求用于请求在图库311的应用进程与视频编辑314的应用进程之间创建编辑会话(session)。因此，基于编辑会话请求的请求数据，视频编辑314可以与图库311所在进程之间建立起编辑会话(session)。As described above, the edit session request is used to request the creation of an edit session between the application process of gallery 311 and the application process of video editing 314 . Therefore, based on the request data of the edit session request, the video editor 314 can establish an edit session with the process in which the gallery 311 is located.
如上所述，编辑能力请求用于获取视频编辑应用的编辑能力。因此，基于编辑能力请求的请求数据，视频编辑314可以基于视频编辑314所具有的编辑能力服务初始化编辑操作环境，例如创建时间轴(Timelines)等基本工程环境，作为编辑环境。完成编辑环境的创建后，视频编辑314所在进程可以向图库311所在进程返回完成创建的返回值，具体可以参考下文中的详细描述，在此不做赘述。As described above, the editing capability request is used to obtain the editing capability of the video editing application. Therefore, based on the request data of the editing capability request, the video editor 314 can initialize the editing operation environment based on the editing capability service possessed by the video editor 314, for example, create a basic engineering environment such as Timelines as the editing environment. After the creation of the editing environment is completed, the process in which the video editor 314 is located may return the return value of the completion of the creation to the process in which the gallery 311 is located. For details, please refer to the detailed description below, which will not be repeated here.
可以理解，在另一些实施例中，上述编辑启动请求也可以为第三请求，该第三请求包括上述编辑会话请求和编辑能力请求所请求的内容，即该第三请求可以用于请求在图库311的应用进程与视频编辑314的应用进程之间创建编辑会话、以及请求获取视频编辑应用的编辑能力。在此不做限制。It can be understood that, in some other embodiments, the above-mentioned editing start request may also be a third request, and the third request includes the content requested by the above-mentioned editing session request and editing capability request, that is, the third request can be used to request the content of the editing session request and the editing capability request. An editing session is created between the application process of 311 and the application process of video editing 314, and an editing capability of the video editing application is requested. There is no restriction here.
在编辑环境就绪之后，视频编辑314可以基于目标对象的文件路径信息加载目标对象至就绪的编辑环境中。在另一些实施例中，视频编辑314也可以基于目标对象的文件路径获取目标对象的相关参数并添加至就绪的编辑环境中，在此不做限制。After the editing environment is ready, the video editor 314 can load the target object into the ready editing environment based on the file path information of the target object. In other embodiments, the video editor 314 may also obtain relevant parameters of the target object based on the file path of the target object and add them to the ready editing environment, which is not limited herein.
404：视频编辑314返回编辑能力列表给图库311。404 : The video editing 314 returns the editing capability list to the gallery 311 .
示例性地，视频编辑314完成编辑操作环境的初始化后，向图库311返回可供图库311调用的编辑能力列表。该编辑能力列表例如可以包括以下编辑能力的一项或多项：分割、变速(或快慢速)、音量、动画、滤镜、单帧导出、裁剪、蒙版、色度抠图、镜像、倒放、定格、画中画、特效、贴纸、添加音乐/音效以及添加字幕等，在此不做限制。Exemplarily, after the video editor 314 completes the initialization of the editing operating environment, it returns to the gallery 311 a list of editing capabilities that can be invoked by the gallery 311 . The list of editing capabilities may include, for example, one or more of the following editing capabilities: split, variable speed (or fast and slow), volume, animation, filter, single frame export, crop, mask, chroma keying, mirroring, Reverse playback, freeze motion, picture-in-picture, special effects, stickers, adding music/sound effects, and adding subtitles, etc., are not limited here.
在一些实施例中，待编辑的目标对象的编码格式不同，视频编辑314向图库311返回的编辑能力列表可以相应的有所不同。例如，当图库311启动编辑的目标对象是MP4格式的视频文件时，视频编辑314可以向图库311返回第一编辑能力列表；当图库311启动编辑的目标对象是AVI格式的视频文件时，视频编辑314可以向图库311返回第二编辑能力列表。其中，第一编辑能力列表与第二编辑能力列表所包括的编辑能力类型和/或各编辑能力类型中的风格选项等可以不同。In some embodiments, the encoding format of the target object to be edited is different, and the editing capability list returned by the video editor 314 to the gallery 311 may be different accordingly. For example, when the target object that the gallery 311 starts to edit is a video file in MP4 format, the video editor 314 can return the first editing capability list to the gallery 311; when the target object that the gallery 311 starts to edit is a video file in AVI format, the video editor 314 314 may return a second list of editing capabilities to gallery 311 . The editing capability types included in the first editing capability list and the second editing capability list and/or the style options in each editing capability type may be different.
在另一些实施例中，不同的应用程序向视频编辑314发送编辑启动请求，视频编辑314所返回的编辑能力列表也会有所不同。例如，在另一些实施例中，AOD 312执行上述步骤402向视频编辑314发送编辑启动请求，则视频编辑314可以向AOD 312返回第三编辑能力列表。该第三编辑能力列表，可以相对于上述第一编辑能力列表或第二编辑能力列表，所包含的编辑能力类型可以更少。In other embodiments, different applications send editing start requests to the video editor 314, and the list of editing capabilities returned by the video editor 314 will also be different. For example, in other embodiments, the
405：图库311基于所返回的编辑能力列表，显示编辑功能控件。405: The gallery 311 displays editing function controls based on the returned editing capability list.
示例性地，图库311基于视频编辑314返回的编辑能力列表，可以在原本的视频浏览界面上显示相应编辑能力对应的编辑功能控件，这些编辑功能控件例如是下述图6a至图6c所示的编辑功能按钮，这些编辑功能控件例如可以显示上述图5c所示编辑按钮531所在菜单栏(作为一级菜单栏)的子菜单栏(或称二级菜单栏)中。可以理解，当编辑能力列表所包括的编辑能力项目较多时，图库311的相应视频浏览界面上还可以显示多级菜单，例如将对应于各编辑能力的编辑功能按钮分类显示为一级菜单栏中编辑按钮531调出的二级菜单栏、三级菜单栏等，在此不做限制。Exemplarily, the gallery 311 can display the editing function controls corresponding to the corresponding editing capabilities on the original video browsing interface based on the editing capability list returned by the video editing 314. These editing function controls are, for example, shown in the following Figures 6a to 6c. Editing function buttons, for example, these editing function controls can be displayed in the sub-menu bar (or the second-level menu bar) of the menu bar (as the first-level menu bar) where the
图6a至图6d示出了手机100显示视频编辑界面的一些UI界面示意图。6a to 6d show some UI interface schematic diagrams of the
用户在上述图5c所示的视频浏览界面530上点击编辑按钮531，手机100屏幕可以显示图6a所示的视频编辑界面610，该视频编辑界面610与上述图2b所示的视频编辑界面220相同，用户可以在视频编辑界面610上点击编辑功能菜单栏611中的任意编辑功能按钮进行编辑操作，用户进行编辑操作后也可以点击视频编辑界面610上的保存按钮612保存已完成的编辑操作。The user clicks the
在另一些实施例中，如果手机100所显示的视频浏览界面是上述图5e所示的视频浏览界面550，则用户在视频浏览界面550上点击编辑按钮，手机100可以显示图6b所示的视频编辑界面620，如图6b所示，如果用户在一级功能菜单栏621中点击滤镜功能按钮，视频编辑界面620上可以显示二级功能菜单栏622，则用户可以进一步点击二级功能菜单栏622中的“添加滤镜”或“添加调节”功能选项进行相应的编辑操作。In other embodiments, if the video browsing interface displayed on the
在另一些实施例中，如果手机100所显示的视频浏览界面是上述图5f所示的视频浏览界面560，则用户在视频浏览界面560上点击编辑按钮，手机100可以显示图6c所示的视频编辑界面630；如果手机100所显示的视频浏览界面是上述图5g所示的视频浏览界面570，则用户在视频浏览界面570上点击风格按钮571，手机100可以显示图6d所示的视频编辑界面640，用户可以在视频编辑界面640上点击一个视频风格选项进行切换至相应视频风格的编辑操作。在此不做限制。In other embodiments, if the video browsing interface displayed on the
406：图库311检测到用户对目标对象的编辑操作，生成相应的编辑操作指令。406: The gallery 311 detects an editing operation of the target object by the user, and generates a corresponding editing operation instruction.
示例性地，手机100所运行的图库311在检测到用户在上述图6a至图6d所示界面上进行的编辑操作后，例如用户在上述图6a至图6c所示界面上点击编辑功能按钮的操作，或者用户在上述图6d所示界面上点击某个视频风格选项的操作等，可以生成相应的编辑操作指令。图库311可将所生成的编辑操作指令可以发送给视频编辑314。可以理解，手机100所运行的图库311生成的编辑操作指令可以通过跨进程通信从图库311所在进程发送至视频编辑314所在进程中进行处理，具体将在下文详细描述，在此不再赘述。Exemplarily, after the gallery 311 running on the
407：视频编辑314基于所接收的编辑操作指令，调用相应的编辑能力对目标对象进行相应编辑处理。407: The video editor 314 invokes the corresponding editing capability to perform corresponding editing processing on the target object based on the received editing operation instruction.
示例性地，手机100所运行的视频编辑314在接收到图库311发来的编辑操作指令后，调用与该编辑操作指令对应的编辑能力，在就绪的编辑环境中对目标对象进行相应的编辑处理。其中，在就绪的编辑环境中对目标对象进行编辑处理，可以包括通过上述步骤403中所创建的时间轴(Timelines)来控制在目标对象上进行编辑处理的时间节点等。例如，视频编辑314所接收的编辑操作指令是滤镜指令，则视频编辑314可以调用滤镜处理能力对当前正在编辑的视频进行滤镜处理。该处理过程中，视频编辑314还可以通过时间轴(Timelines)控制使该滤镜处理作用于正在编辑的视频中的某一段时长的视频片段上。具体可以参考下文详细描述，在此不做赘述。Exemplarily, after receiving the editing operation instruction sent by the gallery 311, the video editor 314 running on the
可以理解，用户如果在图库311的视频编辑界面上进行添加音乐或音效的操作，视频编辑314还可以在当前的编辑环境中调用创建音频轨道能力，并将用户所选的音乐或音效片段添加到所创建的音频轨道上，从而完成对正在处理的视频数据添加音乐或音效的编辑处理过程。具体可以参考下文中详细描述，在此不做赘述。It can be understood that if the user performs an operation of adding music or sound effects on the video editing interface of the gallery 311, the video editor 314 can also invoke the ability to create an audio track in the current editing environment, and add the music or sound effect clips selected by the user to the video editor 314. The created audio track completes the editing process of adding music or sound effects to the video data being processed. For details, reference may be made to the following detailed description, which is not repeated here.
可以理解，在另一些实施例中，视频编辑314基于所接收的编辑操作指令，在当前正在编辑的目标对象数据中添加该编辑操作指令对应的例如滤镜风格之类的选项标签。例如，视频编辑314基于所接收的滤镜指令，可以在当前正在编辑的视频数据上添加用户所选滤镜风格相对应的选项标签，以完成该视频数据的滤镜处理。在此不做限制。It can be understood that, in other embodiments, based on the received editing operation instruction, the video editor 314 adds an option tag, such as a filter style, corresponding to the editing operation instruction in the target object data currently being edited. For example, based on the received filter instruction, the video editor 314 may add an option tag corresponding to the filter style selected by the user to the video data currently being edited, so as to complete the filter processing of the video data. There is no restriction here.
408：视频编辑314将完成编辑处理的返回值发送图库311。408: The video editor 314 sends the return value of the editing process to the gallery 311.
示例性地，视频编辑314基于相应的编辑能力，完成对目标对象的编辑处理后，可以向图库311返回完成编辑处理的返回值。Exemplarily, based on the corresponding editing capability, after completing the editing process on the target object, the video editor 314 may return a return value of the completion of the editing process to the gallery 311 .
409：图库311的编辑界面上显示对应于编辑处理结果的预览效果。409: A preview effect corresponding to the editing processing result is displayed on the editing interface of the gallery 311.
示例性地，图库311接收到视频编辑314编辑成功的返回值(即上述步骤408)后，可以将上述步骤407中视频编辑314编辑处理得到的视频数据显示在当前图库311的编辑界面中，从而为用户呈现编辑处理后视频数据的预览效果。可以理解，手机100屏幕当前显示的图库311的编辑界面，例如可以是上述图6a至图6d所示的视频编辑界面中的任一种编辑界面。Exemplarily, after the gallery 311 receives the return value of the successful editing of the video editing 314 (that is, the above-mentioned step 408), the video data obtained by the editing and processing of the video editing 314 in the above-mentioned
例如，视频编辑314对目标视频进行的编辑处理为添加滤镜、添加贴纸、字幕等，则手机100可以在当前显示的图库311的界面播放处理后的视频数据，以显示上述编辑处理的预览效果。相应地，该预览效果可以包括不同滤镜风格的预览效果、视频画面中添加了贴纸的预览效果、以及视频画面中添加了字幕的预览效果等。For example, if the editing process performed by the video editor 314 on the target video is adding filters, adding stickers, subtitles, etc., the
又例如，视频编辑314对目标视频进行的编辑处理为变速、添加音频/音效等，则手机100也可以在当前显示的图库311的编辑界面上播放处理后的视频数据，以显示上述编辑处理的预览效果。相应地，该预览效果可以包括变速处理后的视频声音播放效果、以及视频数据中添加了音频/音效后的声音播放效果等。For another example, if the editing process performed by the video editor 314 on the target video is variable speed, adding audio/sound effects, etc., the
可以理解，如果用户操作手机100对目标对象进行多次编辑操作，则手机100可以重复执行上述步骤406至409完成相应的编辑处理过程，在此不做限制。It can be understood that if the user operates the
410：图库311检测到用户的保存操作，向视频编辑314发送保存指令。410 : The gallery 311 detects the user's save operation, and sends a save instruction to the video editor 314 .
示例性地，手机100所运行的图库311在检测到用户的保存操作，例如用户在手机100所显示的视频编辑界面上点击保存按钮的操作，图库311可以生成相应的保存指令，发送给视频编辑314进行保存处理。可以理解，手机100所运行的图库311生成的保存指令可以通过跨进程通信从图库311所在进程发送给视频编辑314所在进程进行处理，具体将在下文详细描述，在此不再赘述。Exemplarily, when the gallery 311 running on the
可以理解，在另一些实施例中，用户的保存操作也可以通过其他操作方式完成，例如用户也可以在手机100显示的视频编辑界面上点击分享按钮，以触发对完成编辑的目标对象的保存，图库311检测到用户点击分享按钮的操作也可以生成相应保存指令发送给视频编辑314。在此不做限制。It can be understood that, in other embodiments, the user's saving operation can also be completed by other operation methods, for example, the user can also click the share button on the video editing interface displayed on the
411：视频编辑314基于所接收的保存指令，保存完成编辑的目标对象。411: The video editor 314 saves the edited target object based on the received save instruction.
示例性地，手机100所运行的视频编辑314在接收到保存指令后，可以将完成编辑的目标对象保存至公共目录下。在另一些实施例中，视频编辑314也可以将完成编辑的目标对象保存到与图库311的共用目录下，以供图库311获取，在此不做限制。可以理解，完成编辑的目标对象例如可以是基于图像或视频处理得到的短视频，在此不做限制。Exemplarily, after receiving the save instruction, the video editor 314 running on the
412：图库311获取并显示完成编辑的目标对象。412: The gallery 311 acquires and displays the edited target object.
示例性地，手机100所运行的图库311可以从公共目录中获取完成编辑的目标对象，并显示在更新显示的视频浏览界面上，更新显示的视频浏览界面可以参考上述图2d所示的视频浏览界面260。Exemplarily, the gallery 311 run by the
在另一些实施例中，如果完成编辑的目标对象保存在图库311与视频编辑314的共用目录下，图库311则可以从该共用目录中获取完成编辑的目标对象进行显示，在此不做限制。In other embodiments, if the edited target object is stored in the common directory of the gallery 311 and the video editor 314, the gallery 311 can obtain the edited target object from the shared directory for display, which is not limited herein.
上述步骤401至412中，手机100通过所运行的图库311与视频编辑314之间的交互实现对目标对象的跨进程编辑。手机100在实施上述步骤401至412的过程中始终显示的是图库311的各个界面，包括视频浏览界面以及视频编辑界面等，因此手机100所显示的界面切换过程在用户的视觉中会更加流畅，手机100无需在不同应用的界面之间切换显示，界面显示的响应速度也会更快，用户在操作手机100进行音/视频编辑操作也相应更加方便、快捷。In the
下面结合附图详细介绍手机100实施本申请实施例所提供的多媒体编辑方法的过程中，涉及手机100系统内的两个应用进程之间进行数据、指令的传送以及数据处理指令执行等方面的底层实现逻辑。The process of implementing the multimedia editing method provided by the embodiments of the present application on the
图7根据本申请实施例示出了一种多媒体编辑方法的实施过程中，手机100系统运行的两个应用进程之间的交互时序示意图。FIG. 7 shows a schematic diagram of the interaction sequence between two application processes running in the
如图7所示，手机100运行应用程序的进程包括应用进程01和应用进程02，其中应用进程01例如是上述图库311所在进程，或者其他音/视频应用程序所在的进程；应用进程02例如是上述视频编辑314所在进程，该视频编辑314例如可以是剪映TM、抖音TM、花瓣剪辑TM等具有音/视频编辑能力的应用程序，在此不做限制。其中，应用进程01提供展示目标对象以及编辑处理效果的界面通过手机100的屏幕显示，应用进程02则提供编辑能力服务对应用进程01指定的目标对象进行相应的编辑处理。As shown in FIG. 7 , the process of running the application program on the
参考图7所示，手机100在检测到对目标对象启动编辑的操作后，应用进程01可以调用相关显示资源显示启动编辑的初始化界面710，该初始化界面例如是上述图6a至图6d所示的视频编辑界面。Referring to FIG. 7 , after the
可以理解，手机100在显示初始化界面710的过程中，应用进程01可以向应用进程02提供目标对象的文件路径；应用进程01还可以从应用进程02获取编辑能力支持；应用进程01还可以调用相关显示资源创建界面画布(Surface)，并将所创建的界面画布的地址(或称句柄)通过系统的共享内存能力共享给应用进程02。如此，应用进程01基于应用进程02所支持的编辑能力生成的编辑功能控件、以及在应用进程02中对目标对象的编辑处理效果预览以及缓存的编辑处理后的目标对象相关数据等，均可以显示在应用进程01所运行应用的编辑界面上。应用进程02可以基于应用进程01提供的文件路径获取目标对象，例如目标对象是视频文件，则应用进程02可以对获取的视频文件进行解码，得到可以被编辑的视频文件格式，例如解码得到的视频文件格式为RGB大拇指及单帧图像格式。It can be understood that in the process of displaying the initialization interface 710 on the
可以理解，手机100在检测到用户在初始化界面710上对目标对象进行编辑操作后，应用进程01可从初始化界面710跳转至编辑界面720。用户在编辑界面720上的编辑操作可以触发应用进程01生成相应的编辑操作指令发送给应用进程02。所生成的编辑操作指令例如包括创建轨道指令、编辑轨道指令等，轨道包括视频轨道、音频轨道以及字幕轨道等，具体将在下文详细描述，在此不做限制。It can be understood that after the
仍如图7所示，应用进程02接收到应用进程01发送的编辑操作指令后，可执行对应的创建轨道指令、编辑轨道指令等，创建视频轨道、音频轨道等。所创建的视频轨道、音频轨道等可以采用多轨框架，例如视频轨道可以包括贴纸、花字、特效、滤镜、美颜、倒序等多个编辑子轨道，用于合成视频特效及字幕等界面效果；所创建的音频轨道可以包括音频、音效等编辑子轨道，用于合成混音特效等声音效果。应用进程02执行编辑轨道指令完成相应轨道上的编辑处理过程，处理得到的界面效果和/或声音效果可以同步显示在应用进程01所创建的界面画布上。此时，编辑界面720还可以包括编辑处理效果的预览界面730，预览界面730同步显示编辑处理后的视频画面。Still as shown in FIG. 7 , after receiving the editing operation instruction sent by the application process 01, the application process 02 can execute the corresponding create track instruction, edit track instruction, etc. to create a video track, an audio track, and the like. The created video track, audio track, etc. can use a multi-track framework. For example, a video track can include multiple editing sub-tracks such as stickers, flower characters, special effects, filters, beauty, reverse order, etc., which are used to synthesize video effects and subtitles. Effects; the created audio track can include editing sub-tracks such as audio, sound effects, etc., for synthesizing sound effects such as mixing effects. The application process 02 executes the editing track instruction to complete the editing process on the corresponding track, and the interface effects and/or sound effects obtained from the processing can be displayed on the interface canvas created by the application process 01 synchronously. At this time, the editing interface 720 may further include a
完成编辑操作后，用户在手机100所显示的编辑界面720或者说是预览界面730上进行保存操作后，应用进程01可以向应用进程02发送保存指令，并调用系统相关显示资源在手机100的屏幕上显示保存界面740。上述保存操作例如是用户在上述图2b所示界面上点击保存按钮240的操作，保存界面740例如是上述图2d所示的显示有已保存的完成编辑的视频浏览界面260。After completing the editing operation, after the user performs the saving operation on the editing interface 720 or the
而应用进程02在接收到应用进程01的保存指令后，可以将合成后的视频特效及字幕、混音特效等打包。然后进行编码合并保存处理，即将完成编辑的目标对象进行编码、合并视频特效、字幕以及混音特效生成完成编辑的目标对象。最后将完成编辑的目标对象保存至系统的公共目录下，或者保存至与应用进程01的共用目录下的过程。After the application process 02 receives the save instruction of the application process 01, it can package the synthesized video special effects, subtitles, sound mixing special effects, and the like. Then, the encoding, merging, and saving process is performed, that is, encoding the edited target object, merging video effects, subtitles, and sound mixing effects to generate the edited target object. Finally, the process of saving the edited target object to the public directory of the system, or to the directory shared with the application process 01.
图8a至图8c根据本申请实施例示出了一种多媒体编辑方法的实施过程中，手机100系统运行的两个应用进程之间的具体交互流程示意图。8a to 8c are schematic diagrams showing a specific interaction flow between two application processes running in the system of the
图8a所示为应用进程01通过应用进程02上的编辑能力服务，实现对目标对象进行编辑处理的交互流程，该流程包括以下步骤：Figure 8a shows the interactive flow of the application process 01 implementing the editing process on the target object through the editing capability service on the application process 02, and the flow includes the following steps:
801：应用进程01向应用进程02请求创建编辑会话，请求初始化编辑服务。应用进程02则可以基于请求，创建时间轴(Timelines)等基本工程环境，即编辑环境。完成编辑环境的创建后应用进程02可以向应用进程01发送创建完成的返回值。801: The application process 01 requests the application process 02 to create an editing session, and requests to initialize the editing service. The application process 02 may, based on the request, create a basic engineering environment such as Timelines, that is, an editing environment. After the creation of the editing environment is completed, the application process 02 may send the creation completion return value to the application process 01 .
示例性地，用户(即操作者，Actor)在应用进程01所运行的图库311的浏览界面上进行启动对目标对象编辑的操作后，应用进程01上执行相应的操作指令，向应用进程02请求创建编辑会话(Session)用于建立与应用进程02之间的跨进程通信交互，基于跨进程通信应用进程01可以向应用进程02提供文件路径，例如向应用进程02提供在公共目录下的文件路径，另外还可以授权应用进程01所运行应用程序的私有目录访问权限给应用进程02，以向应用进程02提供编辑所需文件、素材、数据等。Exemplarily, after the user (ie, the operator, Actor) starts the operation of editing the target object on the browsing interface of the gallery 311 run by the application process 01, the application process 01 executes the corresponding operation instruction to request the application process 02. Creating an editing session (Session) is used to establish cross-process communication interaction with the application process 02. Based on the cross-process communication, the application process 01 can provide the application process 02 with a file path, such as providing the application process 02 with a file path in a public directory In addition, the application process 01 can also authorize the private directory access rights of the application program run by the application process 01 to the application process 02, so as to provide the application process 02 with files, materials, data, etc. required for editing.
另外，应用进程01还向应用进程02请求初始化编辑服务，即请求应用进程02创建编辑环境，例如以使应用进程02所运行的视频编辑314提供编辑能力服务。In addition, the application process 01 also requests the application process 02 to initialize the editing service, that is, request the application process 02 to create an editing environment, for example, so that the video editor 314 run by the application process 02 provides editing capability services.
802：应用进程01向应用进程02请求获取编辑能力支持。应用进程02则基于请求向应用进程01返回可提供的编辑能力列表(List)。802: The application process 01 requests the application process 02 to obtain editing capability support. The application process 02 returns a list of available editing capabilities (List) to the application process 01 based on the request.
示例性地，应用进程01向应用进程02发送请求，请求获取应用进程02所运行的视频编辑应用的编辑能力支持。应用进程02基于请求向应用进程01返回可提供的编辑能力列表。可以理解，编辑能力列表可以包括应用进程02所运行的视频编辑314具有的全部或部分编辑能力，在此不做限制。Exemplarily, the application process 01 sends a request to the application process 02 to request to obtain the editing capability support of the video editing application run by the application process 02 . The application process 02 returns a list of available editing capabilities to the application process 01 based on the request. It can be understood that the editing capability list may include all or part of the editing capabilities possessed by the video editor 314 run by the application process 02, which is not limited herein.
基于应用进程02所返回的编辑能力列表，应用进程01可以调用手机100的相关显示资源显示上述初始化界面710，具体参考上述图7中相关描述，在此不再赘述。Based on the editing capability list returned by the application process 02, the application process 01 can call the relevant display resources of the
803：应用进程01向应用进程02发送创建视频轨道(Track)的指令。应用进程02则基于指令创建视频轨道，创建完成后，应用进程02可以向应用进程01发送创建完成的返回值。803: The application process 01 sends an instruction to create a video track (Track) to the application process 02. The application process 02 creates the video track based on the instruction, and after the creation is completed, the application process 02 can send the creation completion return value to the application process 01 .
示例性地，应用进程01所运行的图库311如果检测到用户在界面上操作添加视频素材片段，则应用进程01相应地向应用进程02发送创建视频轨道的指令。应用进程02则基于指令，完成视频轨道的创建。Exemplarily, if the gallery 311 run by the application process 01 detects that the user operates to add a video clip on the interface, the application process 01 sends an instruction to create a video track to the application process 02 accordingly. The application process 02 completes the creation of the video track based on the instruction.
可以理解，视频轨道即是添加视频素材片段的位置，简称视频轨，所有被视频编辑314支持解析的视频素材片段格式可以加载到视频轨中进行编辑以及添加效果等。It can be understood that a video track is the location where video material clips are added, referred to as a video track, and all video clip formats supported and parsed by the video editor 314 can be loaded into the video track for editing and adding effects.
804：应用进程01向应用进程02发送向视频轨道添加视频素材片段(Segment)的指令。应用进程02则基于指令向视频轨道添加视频素材片段，添加完成后，应用进程02可以向应用进程01发送添加完成的返回值。804 : The application process 01 sends an instruction to add a video material segment (Segment) to the video track to the application process 02 . The application process 02 adds video material clips to the video track based on the instruction, and after the addition is completed, the application process 02 can send a return value of the addition completion to the application process 01 .
示例性地，应用进程01可以将用户选择添加的视频素材片段素材的文件路径编辑成添加视频素材片段的指令，发送给应用进程02。Exemplarily, the application process 01 may edit the file path of the video clip material selected by the user to add into an instruction for adding a video clip, and send it to the application process 02 .
应用进程02则基于指令向视频轨道添加视频素材片段，所添加的视频素材片段可以基于应用进程01提供的文件路径获取。The application process 02 adds video material segments to the video track based on the instruction, and the added video material segments can be obtained based on the file path provided by the application process 01 .
805：应用进程01向应用进程02发送创建音频轨道(Track)的指令。应用进程02则基于指令创建音频轨道，创建完成后，应用进程02可以向应用进程01发送创建完成的返回值。805: The application process 01 sends an instruction to create an audio track (Track) to the application process 02. The application process 02 creates an audio track based on the instruction, and after the creation is completed, the application process 02 can send a return value of the creation completion to the application process 01 .
示例性地，应用进程01所运行的图库311如果检测到用户在界面上操作添加音频素材片段，该音频素材片段包括音效素材，则应用进程01相应地向应用进程02发送创建音频轨道的指令，应用进程02则基于指令，完成音频轨道的创建。Exemplarily, if the gallery 311 run by the application process 01 detects that the user operates to add audio material clips on the interface, and the audio material clips include sound effect materials, the application process 01 sends an instruction to create an audio track to the application process 02 accordingly, The application process 02 completes the creation of the audio track based on the instruction.
可以理解，音频轨道即是添加音频素材片段的位置，简称音频轨。It can be understood that an audio track is the location where audio material clips are added, which is referred to as an audio track.
806：应用进程01向应用进程02发送向音频轨道添加音频素材片段(Segment)的指令。应用进程02则基于指令向音频轨道添加音频素材片段，添加完成后，应用进程02可以向应用进程01发送添加完成的返回值。806: The application process 01 sends an instruction to add an audio material segment (Segment) to the audio track to the application process 02. The application process 02 adds audio material clips to the audio track based on the instruction, and after the addition is completed, the application process 02 can send a return value of the addition completion to the application process 01 .
示例性地，应用进程01可以将用户选择添加的音频素材片段的文件路径编辑成添加音频素材片段的指令，发送给应用进程02。Exemplarily, the application process 01 may edit the file path of the audio material segment selected by the user to add into an instruction for adding an audio material segment, and send it to the application process 02 .
应用进程02则基于指令向音频轨道添加音频素材片段，所添加的音频素材片段可以基于应用进程01提供的文件路径获取。The application process 02 adds audio material clips to the audio track based on the instruction, and the added audio material clips can be obtained based on the file path provided by the application process 01 .
807：应用进程01向应用进程02发送创建字幕轨道(Track)的指令。应用进程02则基于指令创建字幕轨道的创建，创建完成后，应用进程02可以向应用进程01发送创建完成的返回值。807: The application process 01 sends an instruction to create a subtitle track (Track) to the application process 02. The application process 02 creates the creation of the subtitle track based on the instruction. After the creation is completed, the application process 02 can send a return value of the creation completion to the application process 01 .
示例性地，应用进程01所运行的图库311如果检测到用户在界面上操作添加字幕，则应用进程01相应地向应用进程02发送创建字幕轨道的指令，应用进程02则基于指令，完成字幕轨道的创建。Exemplarily, if the gallery 311 run by the application process 01 detects that the user operates to add subtitles on the interface, the application process 01 sends an instruction to create a subtitle track to the application process 02 accordingly, and the application process 02 completes the subtitle track based on the instruction. the creation of.
可以理解，字幕轨道即是添加字幕的位置，简称字幕轨。It can be understood that the subtitle track is the location where subtitles are added, referred to as the subtitle track.
808：应用进程01向应用进程02发送向字幕轨道添加文本片段(Segment)的指令。应用进程02基于指令向字幕轨道添加文本片段，添加完成后，应用进程02可以向应用进程01发送添加完成的返回值。808 : The application process 01 sends an instruction to add a text segment (Segment) to the subtitle track to the application process 02 . The application process 02 adds a text segment to the subtitle track based on the instruction, and after the addition is completed, the application process 02 may send a return value of the addition completion to the application process 01 .
示例性地，应用进程01可以将用户选择添加的文本片段文件路径编辑成添加文本片段的指令，发送给应用进程02。Exemplarily, the application process 01 may edit the file path of the text segment selected by the user to add into an instruction for adding a text segment, and send it to the application process 02 .
应用进程02基于指令向字幕轨道添加文本片段，所添加的文本片段基于应用进程01提供的文件路径获取，在另一些实施例中，应用进程01也可以向应用进程02提供文本输入路径，以获取实时的用户在应用进程01所运行的图库311的编辑界面上输入的文本，在此不做限制。The application process 02 adds a text segment to the subtitle track based on the instruction, and the added text segment is obtained based on the file path provided by the application process 01. In other embodiments, the application process 01 can also provide the application process 02 with a text input path to obtain The text input by the real-time user on the editing interface of the gallery 311 run by the application process 01 is not limited herein.
809：应用进程01向应用进程02发送添加特效(Effect)的指令。应用进程02则基于指令，将创建的特效绑定至视频/音频/字幕轨道或所添加的视频/音频/字幕片段上，添加完成后，应用进程02可以向应用进程01发送添加完成的返回值。809: The application process 01 sends an instruction for adding an effect to the application process 02. The application process 02 binds the created special effects to the video/audio/subtitle track or the added video/audio/subtitle segment based on the instruction. After the addition is completed, the application process 02 can send the added return value to the application process 01. .
示例性地，应用进程01向应用进程02发送添加视频特效、音频特效或者字幕特效的指令，其中视频特效例如包括缩放特效、转场特效等，音频特效例如包括变音特效等，字幕特效例如包括设置发光字体等，在此不做限制。应用进程02则基于指令，创建相应特效，并将所创建的特效绑定至视频/音频/字幕轨道或所添加的视频/音频/字幕片段上。Exemplarily, the application process 01 sends an instruction to add a video special effect, an audio special effect or a subtitle special effect to the application process 02, wherein the video special effect includes, for example, a zooming effect, a transition effect, etc. Setting luminous fonts, etc., is not limited here. The application process 02 creates corresponding special effects based on the instructions, and binds the created special effects to the video/audio/subtitle track or the added video/audio/subtitle segment.
810：应用进程01向应用进程02发送导出编辑工程配置文件的指令。810: The application process 01 sends an instruction to export the editing project configuration file to the application process 02.
示例性地，应用进程01所运行的图库311若检测到用户退出编辑的操作，可以将已经完成的编辑处理进度临时保存，例如通过向应用进程02发送导出编辑工程配置文件的指令，通过编辑工程配置文件来保存已创建的时间轴、轨道以及所添加的片段、特效等编辑处理进度。应用进程02则基于指令，将已创建的时间轴、轨道以及所添加的片段、特效等编辑处理进度保存为编辑工程配置文件。Exemplarily, if the gallery 311 run by the application process 01 detects that the user has exited the editing operation, it can temporarily save the completed editing processing progress, for example, by sending an instruction to export the editing project configuration file to the application process 02, and by editing the project Configuration files to save the created timelines, tracks, and editing progress of added clips, effects, etc. The application process 02 saves the editing processing progress of the created timeline, track, and added clips, special effects, etc., as an editing project configuration file based on the instruction.
可以理解，当应用进程01为图库311所在进程、应用进程02为视频编辑314所在进程时，上述图8a所示的步骤801至810，描述的即为上述图4所示步骤401至404的执行过程中涉及的两个应用进程之间的具体交互实现过程。其中，图8b所示的步骤801至810的执行过程，即由图库311执行上述步骤402向视频编辑314发送编辑启动请求时触发，在此不再赘述。It can be understood that when the application process 01 is the process where the gallery 311 is located, and the application process 02 is the process where the video editing 314 is located, the steps 801 to 810 shown in FIG. 8a above describe the execution of the
可以理解，在本申请实施例提供的多媒体编辑方法的实施过程中，两个应用进程之间执行完上述图8a所示交互步骤之后，如果用户在应用进程01(例如图库311的应用进程)的编辑界面上点击各个编辑功能控件，对目标对象进行具体的编辑操作时，图库311的编辑界面上可以显示对应于各编辑操作的预览效果。下面结合图8b，对用户点击编辑功能控件，图库311的编辑界面上显示相应预览效果所涉及的两个进程之间的具体交互过程进行详细介绍。It can be understood that, in the implementation process of the multimedia editing method provided by the embodiment of the present application, after the above-mentioned interaction steps shown in FIG. 8a are performed between the two application processes, if the user is in the When each editing function control is clicked on the editing interface, and a specific editing operation is performed on the target object, the editing interface of the gallery 311 may display a preview effect corresponding to each editing operation. The specific interaction process between the two processes involved in displaying the corresponding preview effect on the editing interface of the gallery 311 after the user clicks on the editing function control will be described in detail below with reference to FIG. 8b .
图8b所示为应用进程01基于应用进程02中的编辑处理，在应用进程01的编辑界面中展示预览效果的交互流程，该流程包括以下步骤：Figure 8b shows the interactive flow of displaying the preview effect in the editing interface of the application process 01 based on the editing process in the application process 02, and the process includes the following steps:
811：应用进程01创建界面画布(Surface)。811: The application process 01 creates an interface canvas (Surface).
示例性地，手机100所运行的应用进程01在执行上述步骤801至802时，可以调用手机100的相关显示资源创建界面画布(Surface)。具体可以参考上述图7所示及相关描述。Exemplarily, when the application process 01 running on the
812：应用进程01向应用进程02共享界面画布的地址。812: The application process 01 shares the address of the interface canvas with the application process 02.
示例性地，应用进程01将所创建的界面画布的地址(或句柄)通过系统的共享内存能力共享给应用进程02。其中，应用进程01与应用进程02之间通过共享内存能力实现共享画布地址可以参考上述图7所示及相关描述，在此不再赘述。Exemplarily, the application process 01 shares the address (or handle) of the created interface canvas with the application process 02 through the shared memory capability of the system. The shared canvas address between the application process 01 and the application process 02 through the shared memory capability can be referred to as shown in FIG. 7 and related descriptions, and will not be repeated here.
813：应用进程02基于界面画布的地址，创建跨进程渲染环境并与画布绑定。813: The application process 02 creates a cross-process rendering environment based on the address of the interface canvas and binds it to the canvas.
示例性地，应用进程02基于应用进程01共享的界面画布地址，创建跨进程渲染环境并将所创建的跨进程渲染环境与相应的界面画布完成绑定，以将应用进程02在相应画布上编辑处理的画布内容同步显示至应用进程01所运行的图库311的界面。Exemplarily, the application process 02 creates a cross-process rendering environment based on the interface canvas address shared by the application process 01 and completes the binding of the created cross-process rendering environment with the corresponding interface canvas, so as to edit the application process 02 on the corresponding canvas. The processed canvas content is synchronously displayed to the interface of the gallery 311 run by the application process 01 .
其中，创建跨进程渲染环境例如可以通过业务编程语言(Enterprise GenerationLanguage，EGL)来实现，EGL是渲染API和原生窗口系统之间的接口，其中渲染API例如可以是OpenGL(Open Graphics Library)提供的OpenGL ES，在此不做限制。Among them, creating a cross-process rendering environment can be realized by, for example, a business programming language (Enterprise Generation Language, EGL). EGL is an interface between a rendering API and a native window system, and the rendering API can be, for example, OpenGL provided by OpenGL (Open Graphics Library). ES, there is no restriction here.
可以理解，任何OpenGL ES应用程序都必须在开始渲染之前使用EGL执行查询并初始化设备可用的显示器、创建渲染表面、创建渲染上下文等任务。在此不做赘述。Understandably, any OpenGL ES application must use EGL to perform tasks such as querying and initializing the display available to the device, creating rendering surfaces, creating rendering contexts, etc., before starting rendering. I won't go into details here.
814：应用进程02从编辑工程配置文件恢复时间轴、轨道以及所添加的片段、特效等编辑处理进度。恢复完成后，应用进程02可以向应用进程01发送恢复完成的返回值。814: The application process 02 restores the editing processing progress of the timeline, track, and added clips, special effects, etc. from the editing project configuration file. After the restoration is completed, the application process 02 may send the restoration completion return value to the application process 01 .
示例性地，如果用户在应用进程01所运行的图库311的浏览界面上启动编辑操作的目标对象，是经上述步骤801至810(可以包括步骤803至809中的部分或全部)处理过的视频文件，则应用进程01再次执行步骤801至802以及上述步骤812至813后，可以触发应用进程02从已保存的编辑工程配置文件中恢复已创建的时间轴、轨道以及所添加的片段、特效等编辑处理进度。如果用户应用进程01所运行的图库311的浏览界面上启动编辑操作的目标对象，是首次被编辑的视频文件，则应用进程01与应用进程02之间可以通过上述步骤801至810完成创建时间轴、轨道以及所添加的片段、特效等编辑处理进度。在此不做限制。Exemplarily, if the user initiates the editing operation on the browsing interface of the gallery 311 run by the application process 01, the target object is the video processed through the above steps 801 to 810 (which may include part or all of the steps 803 to 809). file, the application process 01 executes steps 801 to 802 and the above steps 812 to 813 again, and can trigger the application process 02 to restore the created timeline, track, and added clips, special effects, etc. from the saved editing project configuration file. Edit processing progress. If the target object for starting the editing operation on the browsing interface of the gallery 311 run by the user application process 01 is the video file edited for the first time, the time axis can be created between the application process 01 and the application process 02 through the above steps 801 to 810 , tracks, and editing progress of added clips, effects, etc. There is no restriction here.
815：应用进程01向应用进程02发送合成混音特效的操作指令。815: The application process 01 sends an operation instruction for synthesizing the sound mixing effect to the application process 02.
示例性地，用户可以在应用进程01所运行的图库311的编辑界面上进行制作混音特效的编辑操作，则应用进程01将相应生成的合成混音特效的操作指令发送给应用进程02。Exemplarily, the user may perform an editing operation of creating a sound mixing effect on the editing interface of the gallery 311 run by the application process 01, and the application process 01 sends the correspondingly generated operation instruction for synthesizing the sound mixing effect to the application process 02.
816：应用进程02解码音频素材片段为可编辑音频格式。816: The application process 02 decodes the audio material segment into an editable audio format.
示例性地，应用进程02基于指令，将添加到音频轨道上的音频素材片段解码为可编辑的音频格式，该可编辑音频格式例如包括脉码调制录音(Pulse Code Modulation，PCM)格式等，在此不做限制。Exemplarily, based on the instruction, the application process 02 decodes the audio material segment added to the audio track into an editable audio format, for example, the editable audio format includes a pulse code modulation (Pulse Code Modulation, PCM) format, etc. This does not limit.
817：应用进程02基于解码得到的可编辑音频格式合成混音特效音频。817 : The application process 02 synthesizes the audio of the mixing effect based on the decoded editable audio format.
示例性地，应用进程02基于解码得到的可编辑音频格式，通过所运行的视频编辑314的相应编辑能力将音频轨道上解码后的音频合成为相应的混音特效音频。Exemplarily, based on the decoded editable audio format, the application process 02 synthesizes the decoded audio on the audio track into the corresponding mixing effect audio through the corresponding editing capability of the running video editor 314 .
818：应用进程02同步混音特效音频以展示混音特效音频试听预览。818: The application process 02 synchronizes the audio of the mixing effect to show the preview of the audio of the mixing effect.
示例性地，应用进程02处理得到的混音特效音频，能够通过上述系统共享内存能力同步到手机100的系统中，此时手机100可以播放应用进程02上处理得到的混音特效音频，作为试听预览，同时，应用进程01所运行的图库311的编辑界面上可以显示混音特效音频试听预览的控制按钮，例如播放进度控制按钮、暂停/播放按钮等，也就是说，应用进程02同步到手机100的系统中的混音特效音频，可以通过应用进程01的声音通道和界面画布展示给用户。Exemplarily, the sound mixing special effect audio obtained by the application process 02 can be synchronized to the system of the
819：应用进程01向应用进程02发送合成视频特效和/或字幕特效的操作指令。819 : The application process 01 sends an operation instruction for synthesizing video special effects and/or subtitle special effects to the application process 02 .
示例性地，用户可以在应用进程01所运行的图库311的编辑界面上进行制作视频特效等编辑操作，应用进程01则将相应生成的合成视频特效的操作指令发送给应用进程02。可以理解，用户也可以在应用进程01所运行的图库311的编辑界面上进行添加字幕特效的操作，应用进程01还可以将相应生成的合成字幕特效的操作指令发送给应用进程02Exemplarily, the user can perform editing operations such as creating video special effects on the editing interface of the gallery 311 run by the application process 01, and the application process 01 sends the correspondingly generated operation instructions for synthesizing video special effects to the application process 02. It can be understood that the user can also perform the operation of adding subtitle special effects on the editing interface of the gallery 311 run by the application process 01, and the application process 01 can also send the corresponding generated operation instructions for synthesizing the subtitle special effects to the application process 02
820：应用进程02解码视频素材片段、特效素材为单帧图像数据。820: The application process 02 decodes the video material segment and the special effect material into single-frame image data.
示例性地，应用进程02基于指令，将添加到视频轨道上的视频素材片段以及相应添加到视频轨道上的特效素材等，解码为单帧图像数据，该单帧图像数据可以是RGB格式或者额YUV格式等，在此不做限制。Exemplarily, based on the instruction, the application process 02 decodes the video material clips added to the video track and the special effect material correspondingly added to the video track into single-frame image data, and the single-frame image data may be in RGB format or YUV format, etc., are not limited here.
821：应用进程02基于解码得到的单帧图像数据合成视频特效。821: The application process 02 synthesizes a video special effect based on the decoded single-frame image data.
示例性地，应用进程02基于解码得到的单帧图像数据，在视频轨道上编辑特效素材和视频素材，以合成视频特效。Exemplarily, the application process 02 edits the special effect material and the video material on the video track based on the decoded single-frame image data to synthesize the video special effect.
其中，合成视频特效涉及的编辑效果合成可以包括但不限于以下几类：Among them, the editing effect synthesis involved in synthesizing video special effects may include but not limited to the following categories:
第一类，纹理贴图，例如将贴纸、花字、特效、字幕等进行合成形成纹理贴图；The first category is texture maps, such as synthesizing stickers, flower characters, special effects, subtitles, etc. to form texture maps;
第二类，滤镜、美颜等，例如通过图像算法处理视频画面达到切换滤镜风格、以及美颜效果等；The second category, filters, beauty, etc., such as processing video images through image algorithms to switch filter styles, and beauty effects, etc.;
第三类，升降格，例如通过抽帧和插帧的方法改变播放速率实现升格(即提高播放速率)或降格(即降低播放速率)；The third category, slow and fast motion, for example, by changing the playback rate by means of frame extraction and frame insertion to achieve up-motion (that is, to increase the playback rate) or down-motion (that is, to reduce the playback rate);
第四类，倒放，例如将视频轨道上解码后的图像数据帧序倒置。The fourth type, reverse playback, for example, reverse the frame sequence of the decoded image data on the video track.
可以理解，应用进程02所接收的指令中如果包括添加字幕的指令，应用进程02还可以将相应文本片段添加字幕轨道上相应位置处。It can be understood that, if the instruction received by the application process 02 includes an instruction for adding subtitles, the application process 02 can also add the corresponding text segment to the corresponding position on the subtitle track.
822：应用进程02将合成特效后的单帧图像数据渲染到显示设备接口。822: The application process 02 renders the single-frame image data after synthesizing the special effect to the display device interface.
示例性地，应用进程02完成视频特效和/或字幕特效合成处理后得到合成特效后的单帧图像数据，此时应用进程02可以将合成特效后的单帧图像数据渲染到显示设备接口，以便于渲染到已绑定的界面画布上。Exemplarily, after the application process 02 completes the video special effect and/or the subtitle special effect synthesis process, the single frame image data after the synthesis effect is obtained, and the application process 02 can render the single frame image data after the synthesis effect to the display device interface, so that for rendering to the bound interface canvas.
可以理解，应用进程02所调用的显示设备接口例如是EGL Display，其中Display在本申请实施例中可以理解界面画布的地址或句柄。It can be understood that the display device interface called by the application process 02 is, for example, EGL Display, where Display can understand the address or handle of the interface canvas in this embodiment of the present application.
823：应用进程02向应用进程01同步视频特效和/或字幕特效，以展示视频特效和/或字幕特效界面预览。823: The application process 02 synchronizes the video special effect and/or the subtitle special effect with the application process 01, so as to display a preview of the video special effect and/or the subtitle special effect interface.
示例性地，应用进程02可以通过上述共享内存能力同步视频特效和/或字幕特效至应用进程01的界面画布，进而应用进程01所运行图库311的界面可以显示相应的视频特效和/或字幕特效预览。Exemplarily, the application process 02 can synchronize the video effects and/or subtitle effects to the interface canvas of the application process 01 through the above-mentioned shared memory capability, and then the interface of the gallery 311 run by the application process 01 can display the corresponding video effects and/or subtitle effects. Preview.
可以理解，当应用进程01为图库311所在进程、应用进程02为视频编辑314所在进程时，上述图8b所示的步骤815至823，描述的即为上述图4所示步骤405至409的执行过程中涉及的两个应用进程之间的具体交互实现过程。其中，图8b所示的步骤815和819的执行过程，可以由图库311执行上述步骤406检测到用户对目标对象的编辑操作时触发，在此不再赘述。It can be understood that when the application process 01 is the process where the gallery 311 is located, and the application process 02 is the process where the video editing 314 is located, the steps 815 to 823 shown in the above FIG. 8 b describe the execution of the
另外，图8b所示的步骤811至814，描述的也是上述图4所示步骤401至404的执行过程中涉及的两个应用进程之间的具体交互实现过程，在此不再赘述。In addition, steps 811 to 814 shown in FIG. 8 b also describe the specific interaction implementation process between the two application processes involved in the execution of
可以理解，在本申请实施例提供的多媒体编辑方法的实施过程中，两个应用进程之间执行完上述图8b所示交互步骤之后，如果用户在应用进程01所运行的图库311的编辑界面上点击保存按钮，对编辑处理后的目标对象进行保存操作时，手机100所运行的两个进程之间可以继续执行下文图8c所示各步骤，完成对编辑处理后的目标对象的保存过程。It can be understood that in the implementation process of the multimedia editing method provided by the embodiment of the present application, after the interaction steps shown in FIG. When the save button is clicked to save the edited target object, the steps shown in FIG. 8c can be continued between the two processes running on the
图8c所示为应用进程01通过应用进程02保存完成编辑处理的目标对象的交互流程。可以理解，图8c所示的交互流程中各步骤的执行可以在上述图8a或图8b所示部分步骤前后实施，本申请实施例所提供的多媒体编辑方法的实施步骤也不局限于图8a至图8c所示各步骤以及各步骤的执行顺序，在此不做限制。Figure 8c shows the interaction flow of the application process 01 saving the target object that has completed the editing process through the application process 02. It can be understood that the execution of each step in the interaction process shown in FIG. 8c can be implemented before and after the partial steps shown in FIG. 8a or FIG. 8b, and the implementation steps of the multimedia editing method provided in the embodiment of the present application are not limited to FIGS. 8a to 8b. The steps shown in FIG. 8c and the execution sequence of the steps are not limited herein.
如图8c所示，该流程包括以下步骤：As shown in Figure 8c, the process includes the following steps:
830：应用进程02调用多媒体编/解码器接口，编码混音特效音频，形成音频流文件并缓存。830: The application process 02 invokes the multimedia encoder/decoder interface, encodes the audio with special effects for mixing, and forms an audio stream file and caches it.
示例性地，手机100在执行上述步骤801至809之后、或者执行上述步骤814之后，如果继续执行上述步骤815至818，则可以触发应用进程02执行本步骤830，即调用多媒体编解码接口，对编辑处理得到的混音特效音频进行重新编码，得到音频流文件并缓存，所缓存的音频流文件可以基于系统的共享内存能力同步至应用进程01中，在另一些实施例中，应用进程02缓存编码得到的音频流文件的缓存地址也可以是公共目录，应用进程01也可以通过应用进程02提供的缓存目录同步获取所缓存的音频流文件。其中，编码形成的音频流文件格式例如可以是高级音频编码(Advanced Audio Coding，AAC)格式等，在此不做限制。Exemplarily, after performing the above steps 801 to 809 or after performing the above step 814, if the
可以理解，如果手机100响应于用户操作多次执行上述步骤815至818制作混音特效，手机100所运行的应用进程02可以重复执行本步骤830，在此不做限制。It can be understood that if the
832：应用进程02调用多媒体编/解码器接口，编码合成视频特效后的图像数据，形成视频流文件并缓存。832: The application process 02 invokes the multimedia encoder/decoder interface, encodes the image data after synthesizing video special effects, forms a video stream file and caches it.
示例性地，手机100如果执行上述步骤819至821，则可以触发应用进程02执行本步骤832，即调用多媒体编/解码器接口，对合成视频特效后的图像数据进行重新编码，形成视频流文件并缓存。其中，编码形成的视频流文件格式例如可以是高级视频编码(AdvancedVideo Coding，AVC)格式等，在此不做限制。Exemplarily, if the
可以理解，如果手机100响应于用户操作多次执行上述步骤819至821制作混音特效，手机100所运行的应用进程02可以重复执行本步骤832，在此不做限制。It can be understood that if the
834：应用进程01向应用进程02发送保存指令。834: The application process 01 sends a save instruction to the application process 02.
示例性地，用户可以在应用进程01所运行的图库311的编辑界面上进行保存操作，例如在上述图6a所示的视频编辑界面610上点击保存按钮612的操作，应用进程01则将相应生成的保存指令发送给应用进程02执行。Exemplarily, the user can perform a save operation on the editing interface of the gallery 311 run by the application process 01, for example, click the
836：应用进程02将编码得到的音频流文件和视频流文件进行合并打包形成短视频文件，并保存所生成的短视频文件。836: The application process 02 merges and packages the encoded audio stream file and video stream file to form a short video file, and saves the generated short video file.
示例性地，应用进程02基于所接收的保存指令，将在上述步骤830中缓存的音频流文件和在步骤832中缓存的视频流文件进行合并打包，从而得到合成了音频流文件和视频流文件的短视频文件。其中，应用进程02例如可以通过调用多媒体封装器(Media Muxer)对音频流文件及视频流文件进行打包封装；打包形成的短视频文件格式例如可以是MP4、或音频视频交错格式(Audio Video Interleaved,AVI)等媒体文件格式，在此不做限制。Exemplarily, based on the received save instruction, the application process 02 merges and packs the audio stream file cached in the above-mentioned step 830 and the video stream file cached in the step 832, thereby obtaining a synthesized audio stream file and a video stream file. short video files. Wherein, the application process 02, for example, can package and encapsulate the audio stream file and the video stream file by calling the multimedia encapsulator (Media Muxer); AVI) and other media file formats, which are not limited here.
可以理解，在上述步骤819至821的执行过程中如果包括添加字幕特效的编辑处理过程，本步骤836中应用进程02在打包音频流文件和视频流文件的同时，还可以合成添加了文本片段的字幕轨道，以使形成的短视频文件中包括字幕特效。It can be understood that in the execution process of the above-mentioned steps 819 to 821, if the editing process of adding subtitle special effects is included, in this step 836, the application process 02 can also synthesize the text fragment added while packaging the audio stream file and the video stream file. Subtitle track, so that the formed short video file includes subtitle special effects.
至此可以理解，本申请实施例所提供的多媒体编辑方法是通过跨进程通信的方式实现将应用进程01上接收到的音/视频编辑处理任务交给应用进程02上的编辑能力服务来处理，编辑处理的结果再通过跨进程通信方式同步至应用进程01。上述步骤812、步骤818、步骤823以及步骤830中描述的共享内存能力，也是手机100系统实现跨进程通信所基于的实现方式之一。It can be understood so far that the multimedia editing method provided by the embodiment of the present application is to implement the audio/video editing processing task received on the application process 01 to the editing capability service on the application process 02 by means of cross-process communication. The processing result is then synchronized to the application process 01 through inter-process communication. The shared memory capability described in the above steps 812 , 818 , 823 and 830 is also one of the implementation manners on which the
实施例2Example 2
本申请实施例以音/视频应用是AOD 312为例，介绍本申请的多媒体编辑方法的具体实施过程。The embodiment of the present application takes the audio/video application as
图9根据本申请实施例示出了另一种多媒体编辑方法的实施流程示意图。FIG. 9 is a schematic diagram illustrating an implementation flow of another multimedia editing method according to an embodiment of the present application.
如图9所示，本申请实施例中实施多媒体编辑方法的流程各步骤与上述图4所示流程各步骤的区别仅在于交互主体不同，图9所示流程各步骤涉及的交互主体为AOD 312和视频编辑314。具体地，图9所示流程包括以下步骤：As shown in FIG. 9 , the steps of the process for implementing the multimedia editing method in the embodiment of the present application differ from the steps of the process shown in FIG. 4 above only in that the interaction subject is different. The interaction subject involved in each step of the process shown in FIG. 9 is the
901：AOD 312检测到用户启动对目标对象编辑的操作。901:
902：AOD 312发送编辑启动请求至视频编辑314。其中，该编辑启动请求至少包括获取编辑能力支持的请求数据以及目标对象的文件路径。902 :
903：视频编辑314基于所接收的编辑启动请求，初始化编辑操作环境，并向就绪的编辑操作环境中添加目标对象。903: The video editor 314 initializes the editing operation environment based on the received editing start request, and adds the target object to the ready editing operation environment.
904：视频编辑314返回编辑能力列表给AOD 312。904: The video editing 314 returns the editing capability list to the
905：AOD 312基于所返回的编辑能力列表，显示编辑功能控件。905:
906：AOD 312检测到用户对目标对象的编辑操作，生成相应的编辑操作指令。906: The
908：视频编辑314将完成编辑处理的返回值发送给AOD 312。908: The video editor 314 sends the return value of the completed editing process to the
909：AOD 312的编辑界面上显示对应于编辑处理结果的预览效果。909: A preview effect corresponding to the editing processing result is displayed on the editing interface of the
910：AOD 312检测到用户的保存操作，生成保存指令。910: The
911：视频编辑314基于所接收的保存指令，保存完成编辑的目标对象。911: The video editor 314 saves the edited target object based on the received save instruction.
912：AOD 312获取并显示完成编辑的目标对象。912:
图10a至图10g根据本申请实施例示出了一些UI界面示意图。10a to 10g show some schematic diagrams of UI interfaces according to embodiments of the present application.
如图10a所示，用户可以通过图库311的图片/视频浏览界面1010上点击更多按钮1011并选择“设置为超级壁纸”选项进入图10e所示的AOD 312的超级壁纸编辑界面1030。As shown in Fig. 10a, the user can click the
在另一些实施例中，用户也可以通过手机100系统的设置应用找到超级壁纸选项打开图10b所示的超级壁纸设置界面1020，再点击超级壁纸设置界面1020上的“从图库选择”选项进入图10c所示的AOD 312的编辑界面1030。In other embodiments, the user can also find the super wallpaper option through the setting application of the
如图10c所示，用户可以在编辑界面1030上点击剪辑按钮1031对用户所选的视频素材1032进行剪辑处理，例如可以从视频素材1032中智能选取3s时长视频片段，也可以点击更换素材按钮1032更换当前编辑的视频素材1032。As shown in FIG. 10c , the user can click the
如图10d所示，用户还可以在编辑界面1030上点击滤镜按钮1033对剪辑处理后的视频素材1032的片段进行切换滤镜处理，例如用户点击切换至滤镜风格1034，如此编辑界面1030上便可以显示滤镜风格1034的视频素材1032片段。As shown in FIG. 10d , the user can also click the
如图10e所示，用户可以在编辑界面1030上点击息屏修剪按钮1035，选择修剪形状1036，编辑界面1030上便可以显示与用户所选修剪形状1036相对应的预览壁纸1037。As shown in FIG. 10e , the user can click the
用户在编辑界面1030上完成编辑操作后，可以点击编辑界面1030右上角的保存按钮1038保存编辑操作结果。之后，手机100可以显示图10e所示的壁纸应用界面1040。After the user completes the editing operation on the
如图10e所示，壁纸应用界面1040显示的即为AOD 312完成设置后的预览效果界面，用户可以在图10e所示的壁纸应用界面1040上点击编辑按钮1041返回图10c所示的编辑界面1030，再次进行编辑处理。用户也可以在图10e所示的壁纸应用界面1040上点击应用按钮1042，确定将壁纸应用界面1040所显示的预览效果应用于AOD 312的息屏显示界面上。AOD 312的息屏显示界面可以参考图10g所示，手机100处于息屏状态下，显示经上述图10c至图10e所示界面完成编辑处理的动态视频壁纸。As shown in Fig. 10e, the
可以理解，用户在编辑界面1030上进行的点击剪辑按钮1031、滤镜按钮1033以及息屏修剪按钮1035进行各项编辑操作，可以通过执行上述图9所示的交互流程调用视频剪辑应用314所具有的编辑能力进行编辑处理；用户在编辑界面1030上点击保存按钮1038进行的保存操作，也可以通过上述图9所示的交互流程指令视频剪辑应用314将相应编辑处理的结果保存下来，在此不再赘述。It can be understood that when the user clicks the
下面实施例以图库311和视频编辑314进行跨进程交互举例，对应用程序框架层中的编辑能力服务(例如图3中的编辑能力服务321)的组成结构进行简要介绍。The following embodiment briefly introduces the composition and structure of the editing capability service in the application framework layer (for example, the editing capability service 321 in FIG. 3 ) by taking the example of cross-process interaction between the gallery 311 and the video editing 314 .
图11根据本申请实施例示出了一种编辑能力服务的组成结构示意图。FIG. 11 shows a schematic diagram of the composition structure of an editing capability service according to an embodiment of the present application.
如图11所示，编辑能力服务1100提供编辑能力可以基于媒体展示集成层1110、媒体编辑处理平台1120、以及系统平台1130所提供的功能模块。As shown in FIG. 11 , the editing capability provided by the editing capability service 1100 may be based on the functional modules provided by the media presentation integration layer 1110 , the media editing processing platform 1120 , and the system platform 1130 .
其中，媒体展示集成层1110包括UI界面、能力调用接口以及媒体编辑能力软件开发工具包。UI界面用于向调用编辑能力服务1100的视频编辑314提供编辑能力展示资源，UI界面可以提供素材列表显示以及效果预览界面等展示资源，在本申请实施例中，视频编辑314调用编辑能力服务1100能够提供的编辑能力展示资源可以跨进程同步至请求编辑能力支持的图库311、AOD 312以及相机313等。能力调用接口用于调用媒体编辑处理平台1120所提供的媒体框架能力、渲染能力、音/视频编辑能力媒体编辑处理能力、以及媒体文件解析、媒体文件编解码以及媒体文件播放能力等系统能力。The media presentation integration layer 1110 includes a UI interface, a capability invocation interface, and a media editing capability software development toolkit. The UI interface is used to provide editing capability display resources to the video editor 314 calling the editing capability service 1100. The UI interface can provide display resources such as material list display and effect preview interface. In the embodiment of the present application, the video editor 314 calls the editing capability service 1100. The editing capability display resources that can be provided can be synchronized across processes to the gallery 311 ,
媒体编辑处理平台1120可以包括媒体控制层1121以及能力层1122。其中媒体控制层1121可以包括基于所创建的时间轴(Timelines)实现工程配置、渲染管理、轨道管理、素材管理以及片段管理等管理职能的编辑工程管理；输入的视频流、音频流、图片等数据；用于对输入数据进行格式解析的分离器、解码器等；媒体控制层1121还用于管理音/视频编辑类型，包括能力类的倒序、快慢速等，以及素材类的特效、贴纸、美颜、花字、滤镜等；另外，媒体控制层1121还具有对编辑处理得到的可编辑格式音频以及单帧图像数据等进行编码封装的编码器、封装器等。The media editing processing platform 1120 may include a
能力层1122可以包括开源媒体框架能力、OpenGL渲染能力以及音/视频编辑能力等。The capability layer 1122 may include open source media framework capabilities, OpenGL rendering capabilities, audio/video editing capabilities, and the like.
系统平台1130则可以提供一些系统功能模块，包括媒体文件解析、媒体文件编解码、媒体文件播放器、渲染引擎等，以供上述媒体展示集成层1110的能力调用接口调用。The system platform 1130 may provide some system function modules, including media file parsing, media file encoding and decoding, media file player, rendering engine, etc., for invoking the capability calling interface of the above-mentioned media presentation integration layer 1110 .
可以理解，图11所示并不能构成对编辑能力服务1100的组成结构的限定，手机100上安装的视频编辑314所具备的编辑能力不同时，可以使得手机100能够提供的编辑能力服务1100的组成有所差异，在另一些实施例中，编辑能力服务1100的组成还可以包括比图11所示更多或更少的结构，在此不做限制。It can be understood that what is shown in FIG. 11 does not constitute a limitation on the composition of the editing capability service 1100. When the editing capabilities of the video editor 314 installed on the
图12根据本申请实施例示出了一种手机100的硬件结构示意图。FIG. 12 shows a schematic diagram of the hardware structure of a
手机100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。The
其中，传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and an environment Light sensor 180L, bone conduction sensor 180M, etc.
可以理解的是，本发明实施例示意的结构并不构成对手机100的具体限定。在本申请另一些实施例中，手机100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。It can be understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processingunit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。The
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从上述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。在本申请实施例中，存储器中可以存储有实施本申请实施例所提供的多媒体编辑方法的指令和数据，处理器110可以通过控制器根据指令操作码和时序信号生成操作控制信号，完成指令的执行过程以实现本申请实施例所提供的多媒体编辑方法，在此不做赘述。A memory may also be provided in the
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuitsound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purposeinput/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。In some embodiments, the
在本申请实施例中，用户选择编辑或添加到音频轨道上的音频素材可以通过PCM接口进行抽样、量化和编码后传送，在此不做赘述。其中，PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。上述I2S接口和上述PCM接口都可以用于音频通信。In the embodiment of the present application, the audio material selected to be edited or added to the audio track by the user may be sampled, quantized and encoded through the PCM interface and then transmitted, which will not be described in detail here. Among them, the PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对手机100的结构限定。在本申请另一些实施例中，手机100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。It can be understood that, the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the
充电管理模块140用于从充电器接收充电输入。电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the
手机100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。The wireless communication function of the
天线1和天线2用于发射和接收电磁波信号。移动通信模块150可以提供应用在手机100上的包括2G/3G/4G/5G等无线通信的解决方案。无线通信模块160可以提供应用在手机100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(globalnavigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The
手机100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。The
显示屏194用于显示图像，视频等。显示屏194包括显示面板。在一些实施例中，手机100可以包括1个或N个显示屏194，N为大于1的正整数。Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. In some embodiments, the
手机100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。ISP用于处理摄像头193反馈的数据。The
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，手机100可以包括1个或N个摄像头193，N为大于1的正整数。Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the
视频编解码器用于对数字视频压缩或解压缩。手机100可以支持一种或多种视频编解码器。这样，手机100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。在本申请实施例中，用户选择编辑或添加到视频轨道上的视频素材片段可以通过视频编解码器压缩编码或者解码。在此不做赘述。Video codecs are used to compress or decompress digital video. The
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展手机100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。The
内部存储器121可以用于存储计算机可执行程序代码，该可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储手机100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行手机100的各种功能应用以及数据处理。Internal memory 121 may be used to store computer executable program code, which includes instructions. The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the
手机100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。The
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。The
扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。手机100可以通过扬声器170A收听音乐，或收听免提通话。
受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当手机100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。The
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。手机100可以设置至少一个麦克风170C。在另一些实施例中，手机100可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，手机100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。The
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。The
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。手机100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，手机100根据压力传感器180A检测该触摸操作的强度。手机100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于手机100的表面，与显示屏194所处的位置不同。Touch sensor 180K, also called "touch device". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the
在本申请实施例中，手机100可以通过压力传感器180A、触摸传感器180K检测用户在手机100所显示的图库等应用的编辑界面上针对目标对象(例如视频对象)的编辑操作、保存操作等，在此不做赘述。In this embodiment of the present application, the
示例性的，手机100还可以包括按键190、马达191、指示器192、SIM卡接口195(或eSIM卡)等一项或多项。当然，本申请实施例中用于执行上述多媒体编辑方法的电子设备还可以是平板电脑、PC或可穿戴设备等，本申请实施例对此不作任何限制。Exemplarily, the
在说明书对“一个实施例”或“实施例”的引用意指结合实施例所描述的具体特征、结构或特性被包括在根据本申请实施例公开的至少一个范例实施方案或技术中。说明书中的各个地方的短语“在一个实施例中”的出现不一定全部指代同一个实施例。Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one example embodiment or technique disclosed in accordance with the embodiments of this application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.
本申请实施例的公开还涉及用于执行文本中的操作装置。该装置可以专门处于所要求的目的而构造或者其可以包括被存储在计算机中的计算机程序选择性地激活或者重新配置的通用计算机。这样的计算机程序可以被存储在计算机可读介质中，诸如，但不限于任何类型的盘，包括软盘、光盘、CD-ROM、磁光盘、只读存储器(ROM)、随机存取存储器(RAM)、EPROM、EEPROM、磁或光卡、专用集成电路(ASIC)或者适于存储电子指令的任何类型的介质，并且每个可以被耦合到计算机系统总线。此外，说明书中所提到的计算机可以包括单个处理器或者可以是采用针对增加的计算能力的多个处理器涉及的架构。The disclosure of the embodiments of the present application also relates to a device for performing an operation in a text. This apparatus may be specially constructed for the required purposes or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored on a computer readable medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read only memory (ROM), random access memory (RAM) , EPROM, EEPROM, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processors for increased computing power.
本文所提出的过程和显示器固有地不涉及任何具体计算机或其他装置。各种通用系统也可以与根据本文中的教导的程序一起使用，或者构造更多专用装置以执行一个或多个方法步骤可以证明是方便的。在一下描述中讨论了用于各种这些系统的结构。另外，可以使用足以实现本申请实施例公开的技术和实施方案的任何具体编程语言。各种编程语言可以被用于实施本公开，如本文所讨论的。The processes and displays presented herein are not inherently related to any specific computer or other apparatus. Various general-purpose systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform one or more method steps. Architectures for various of these systems are discussed in the following description. Additionally, any specific programming language sufficient to implement the techniques and implementations disclosed in the examples herein may be used. Various programming languages may be used to implement the present disclosure, as discussed herein.
另外，在本说明书所使用的语言已经主要被选择用于可读性和指导性的目的并且可能未被选择为描绘或限制所公开的主题。因此，本申请实施例公开旨在说明而非限制本文所讨论的概念的范围。Additionally, the language used in this specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or limit the disclosed subject matter. Accordingly, the present disclosure of embodiments is intended to illustrate, but not to limit, the scope of the concepts discussed herein.
Claims (14)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615710.1ACN115016871B (en) | 2021-12-27 | 2021-12-27 | Multimedia editing method, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615710.1ACN115016871B (en) | 2021-12-27 | 2021-12-27 | Multimedia editing method, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115016871Atrue CN115016871A (en) | 2022-09-06 |
| CN115016871B CN115016871B (en) | 2023-05-16 |
Family
ID=83064327
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111615710.1AActiveCN115016871B (en) | 2021-12-27 | 2021-12-27 | Multimedia editing method, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115016871B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320604A (en)* | 2023-02-17 | 2023-06-23 | 北京三快在线科技有限公司 | Multimedia resource processing method, device, equipment and storage medium |
| CN117939231A (en)* | 2022-10-26 | 2024-04-26 | 荣耀终端有限公司 | A media post-processing system, method and electronic device |
| WO2025050885A1 (en)* | 2023-09-05 | 2025-03-13 | 华为技术有限公司 | Cross-application interface display method and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060184980A1 (en)* | 2003-04-07 | 2006-08-17 | Cole David J | Method of enabling an application program running on an electronic device to provide media manipulation capabilities |
| US20140181666A1 (en)* | 2011-08-09 | 2014-06-26 | Stephen Trey Moore | Digital media editing |
| CN110309006A (en)* | 2019-06-28 | 2019-10-08 | 百度在线网络技术（北京）有限公司 | A kind of funcall method, apparatus, terminal device and storage medium |
| CN113391739A (en)* | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Function menu display method and device, electronic equipment and storage medium |
- 2021
- 2021-12-27CNCN202111615710.1Apatent/CN115016871B/enactiveActive
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060184980A1 (en)* | 2003-04-07 | 2006-08-17 | Cole David J | Method of enabling an application program running on an electronic device to provide media manipulation capabilities |
| US20140181666A1 (en)* | 2011-08-09 | 2014-06-26 | Stephen Trey Moore | Digital media editing |
| CN110309006A (en)* | 2019-06-28 | 2019-10-08 | 百度在线网络技术（北京）有限公司 | A kind of funcall method, apparatus, terminal device and storage medium |
| CN113391739A (en)* | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Function menu display method and device, electronic equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117939231A (en)* | 2022-10-26 | 2024-04-26 | 荣耀终端有限公司 | A media post-processing system, method and electronic device |
| CN116320604A (en)* | 2023-02-17 | 2023-06-23 | 北京三快在线科技有限公司 | Multimedia resource processing method, device, equipment and storage medium |
| WO2025050885A1 (en)* | 2023-09-05 | 2025-03-13 | 华为技术有限公司 | Cross-application interface display method and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115016871B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110109636B (en) | Screen projection method, electronic device and system | |
| CN113747047B (en) | A method and device for video playback | |
| CN115016871B (en) | Multimedia editing method, electronic device and storage medium | |
| CN114706664A (en) | Interaction method, electronic device and storage medium for cross-device task processing | |
| CN113835649B (en) | Screen projection method and terminal | |
| CN109819306B (en) | Media file clipping method, electronic device and server | |
| JP6284931B2 (en) | Multiple video playback method and apparatus | |
| CN115223579B (en) | Codec negotiation and switching method | |
| CN116055799B (en) | Multi-track video editing method, graphical user interface and electronic equipment | |
| CN115756268A (en) | Method, device, screen projection system and terminal for cross-device interaction | |
| CN112527174A (en) | Information processing method and electronic equipment | |
| US20240086035A1 (en) | Display Method and Electronic Device | |
| CN115550559B (en) | Video picture display method, device, equipment and storage medium | |
| CN116052701A (en) | Audio processing method and electronic equipment | |
| WO2022179530A1 (en) | Video dubbing method, related device, and computer readable storage medium | |
| CN114339308B (en) | Video stream loading method, electronic device and storage medium | |
| CN118474276A (en) | Video processing method and electronic device | |
| CN115309316B (en) | Device using method and device | |
| CN114449200A (en) | Audio and video call method, device and terminal equipment | |
| CN112565873A (en) | Screen recording method and device, equipment and storage medium | |
| CN117692714B (en) | Video display method, electronic device, computer program product, and storage medium | |
| CN117201955B (en) | Video shooting method, device, equipment and storage medium | |
| CN115515008B (en) | Video processing method, terminal and video processing system | |
| CN113031903B (en) | Electronic equipment and audio stream synthesis method thereof | |
| CN119762373A (en) | Image processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | Address after:Unit 3401, unit a, building 6, Shenye Zhongcheng, No. 8089, Hongli West Road, Donghai community, Xiangmihu street, Futian District, Shenzhen, Guangdong 518040 Patentee after:Honor Terminal Co.,Ltd. Country or region after:China Address before:3401, unit a, building 6, Shenye Zhongcheng, No. 8089, Hongli West Road, Donghai community, Xiangmihu street, Futian District, Shenzhen, Guangdong Patentee before:Honor Device Co.,Ltd. Country or region before:China | |
| CP03 | Change of name, title or address |