US20190272853A1 - User interface for editing digital media objects - Google Patents

Abstract

A computer-implemented method involving an object-based editing environment for defining a digital video as a linear arrangement of discrete media objects, each digital media object electronically stored as a digital file. The method includes displaying a graphical representation of a first digital media object within the editing environment; displaying one or more user interface icons associated with the digital media object; displaying a first menu in response to receiving a first user input; receiving second user input associated with the first menu and identifying a second digital media object; displaying a graphical representation of the second digital media object within the editing environment; and generating a digital video comprising the first digital media object and the second digital media object.

Description

TECHNICAL FIELD
• This specification generally relates to digital media editing particularly for mobile processing devices.
SUMMARY
• In general, innovative aspects of the subject matter described in this specification can be embodied in methods that include the actions of providing an object-based editing environment for defining a digital video as a linear arrangement of discrete media objects, each digital media object electronically stored as a digital file, displaying a graphical representation of a first digital media object within the editing environment, displaying one or more user interface icons associated with the digital media object, receiving first user input, the first user input corresponding to a first interface icon of the one or more interface icons, in response to receiving the first user input, displaying a first menu, receiving second user input associated with the first menu, the second user input identifying a second digital media object, displaying a graphical representation of the second digital media object within the editing environment, and generating a digital video comprising the first digital media object and the second digital media object. Other embodiments of these aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.
• These and other embodiments may each optionally include one or more of the following features. For instance, the first digital media object comprises one of a first digital video, a first digital image, a first title and a first transition; the second digital media object includes one of a second digital video, a second digital image, a second title and a second transition, the first interface icon includes an insert icon indicating insertion of the second media object before or after the first digital media object; generating the digital video includes merging the first and second digital objects; the first menu comprises a plurality of option buttons, each option button corresponding to a different type of media object; the second user input includes a selection of an option button, the second media object being of a type of media object corresponding to the option button; the first menu is a contextual menu, the content of which is based on a type of the first interface icon; the first interface icon is an insertion icon; actions further include: receiving third user input, the third user input corresponding to a second interface icon of the one or more interface icons, and in response to receiving the third user input, displaying a second menu, the second menu is a contextual menu, the content of which is based on a type of the first media object; the first menu is displayed within the graphical representation of the first media object; and actions further include displaying an overview timeline, the overview timeline including thumbnail representations of each of the first and second media objects.
• The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIGS. 1A and 1B illustrate an example mobile processing device.
• FIG. 2 depicts an exemplary process for editing digital media that can be performed by a mobile device application running on the mobile processing device ofFIGS. 1A and 1B.
• FIGS. 3A-3F depict the mobile processing device ofFIGS. 1A and 1B running an example mobile device application.
• Like reference numbers represent corresponding parts throughout.
DETAILED DESCRIPTION
• Implementations of the present disclosure are generally directed to mobile device applications for enabling video editing to be performed on suitable mobile processing devices. Such mobile device applications can provide an object based environment where distinct digital media units (e.g., still images, video clips, audio clips, etc.) can be imported and edited. These mobile device applications can also be operable to merge the imported digital media units based on modifications achieved in the editing environment.
• FIGS. 1A and 1B illustrate an examplemobile processing device100. In the depicted example, themobile processing device100 is provided as a smartphone. It is appreciated, however, implementations of the present disclosure can be realized using tablet computing devices, laptop computers, personal digital assistants (PDAs), mobile phones, or any one of many other known types of suitable mobile processing devices. As shown, themobile processing device100 includes one ormore processors102, memory104 (e.g., computer readable media, read-only memory (ROM), random access memory (RAM), flash memory, a removable memory card (e.g., SD card), a hard disk, or any combination thereof), atransmitter106, areceiver108, a keypad110 (e.g., a soft key), adisplay112, a user-interface (UI) module114 (e.g., a graphical user interface module for touchscreen devices),speakers116, amicrophone118,sensors120, and abattery122. In some examples, theprocessor102 can control each component of themobile processing device100 and can accessmemory104 to runapplications124, and/or retrieveuser data126, which can be stored in the memory. Thetransmitter106 and thereceiver108 transmit and receive signals (e.g., via one or more antennas) to other communication devices, base stations, computers, satellites (e.g., GPS satellites), and/or wireless access points. Thekeypad110, thedisplay112, thespeaker116, and themicrophone118 are input/output devices for communicating with a user. Thebattery120 provides power to appropriate components of the mobile processing device.
• In some examples, theprocessor102 can include adisplay manager128 and aninput manager130. Thedisplay manager128 can be operable to manage displayed on thedisplay112. In some implementations, an operating system running on themobile device102 can use thedisplay manager128 to arbitrate access to thedisplay112 for a number ofapplications124 running on themobile processing device100. In some examples, a number ofapplications124 can run concurrently on themobile processing device100, with each application being shown to the user in respective windows on thedisplay112. In such examples, thedisplay manager128 can control which portions of eachapplication124 are shown on thedisplay112. In some implementations, thedisplay manager128 can control the orientation (i.e., portrait orientation or landscape orientation) of thedisplay112. In some examples, theinput manager130 can be operable control the handling of data received from a user via one or more input devices of themobile processing device100. For example, theinput manager130 can determine whichapplication124 should be invoked as a result of user received input. In some implementations, theUI module114 can coordinate with the input devices to a interpret user input. The interpreted input can be evaluated by theinput manager130 to determine whichapplication124 should be invoked.
• FIG. 2 depicts anexample process200 for editing digital media that can be performed by one or more mobile device applications executed on themobile processing device100. As shown, theprocess200 includes obtaining a plurality of digital media units, atoperation202. Digital media units can be obtained by any one of several suitable procedures. In some examples, a digital media unit can include a digital video file, a digital image file and/or a digital audio file. In some implementations, digital media units can be obtained by accessing theuser data126 stored inmemory104. That is, digital video files, digital image files and/or a digital audio files can be stored inmemory104. In some examples, the mobile device application can enable a user to utilize the input components to select one or more digital media units stored inmemory104. In some examples, the mobile device application can copy the selected digital media units and store them in a suitable directory, or simply reference their location inmemory104. In some implementations, digital media units can also be obtained through a communications link (e.g., a wired or wireless communications link). For example, the mobile device application can enable a user to utilize the input components ofmobile processing device100 to select specific media units for downloading from a remote server or a computing system (e.g., a personal computer communicatively coupled to mobile processing device100). The downloaded digital media units can be stored inmemory104. In some examples, digital media units can be created using and stored to themobile processing device100. For example, a digital camera can be used to capture digital images and/or digital video and store them as digital files on themobile processing device100. As another example, a microphone of themobile processing device100 can capture audio and store it in a digital audio file.
• As shown, theprocess200 also includes providing an editing environment, atoperation204. In some examples, the editing environment is an object based editing environment. In such examples, obtained digital media units can be represented as discrete objects arranged in the editing environment. In some implementations, a number of the discrete objects are arranged linearly by the mobile device application in a prescribed or random order. In some implementations, the mobile device application can enable users to rearrange the objects within the editing environment by utilizing the input components of themobile processing device100. In some examples, the digital media units can be arranged linearly, end-to-end, or in an overlapping manner by users. The mobile device application can also enable users to introduce additional objects to the editing environment. For example, objects representing additional digital media units can be introduced. In some implementations, objects in the editing environment can represent elements other than digital media units. For example, objects can represent effects or actions (such as transitions) to be implemented. Such objects can be imported manually by users, or automatically by the mobile device application.
• As shown, theprocess200 further includes merging the obtained plurality of digital media units to form a substantially continuous digital media unit, atoperation206. For example individual video clips, audio clips, and/or or still images can be merged (merged still images may be provided in the form of a slide show where each image is displayed for a predetermined duration). In some implementations, the plurality of digital media units are stitched together piece-by-piece according to the arranged configuration of representative objects in the editing environment. Further, in some examples, where an object in the editing environment represents a transition effect, the resulting “merge” of the pre-input media and the post-input media for that transition can be created based on the type of transition selected. The amount of overlap can be defined by the duration property for that transition object.
• FIGS. 3A-3F depict themobile processing device100 running an example mobile device application (e.g., anapplication124 stored in memory104). As shown (for example inFIGS. 3A-3C), the mobile device application presents anediting environment302 to users via thedisplay112 of themobile processing device100. A number of discrete virtual objects304 (e.g.,304aand304b) are disposed in a linear configuration within theediting environment302. In this example, the objects304 represent distinct digital media units available frommemory104 of themobile processing device100. In some implementations, the objects304 are movable withinediting environment302. Therefore, as described above, the objects304 can be rearranged in any prescribed order by a user. For example, a “drag-and-drop” technique can be executed by a user to rearrange the objects304.
• As shown, the mobile device application can provide amenu bar306 including a number of selectable menu items. For example, selecting menu item308 (“Projects”) enables a user to access previously saved projects (i.e., instances of editing environment302), or to create new projects. Various projects can include representations of the same, or different, digital media units. Selectingmenu item310 enables the user to adjust any one of several default settings for the project (e.g., screen orientation, video or sound quality, etc.). Selectingmenu item312 enables the user to view a help file. The help file can include various resources such as troubleshooting techniques, software patch or version notes, etc. Selecting menu item314 (“Preview”) enables the user to observe a preliminary version, or preview of a merged digital media unit (as described in greater detail below). In this example, themenu bar306 also includesselectable toggle items316a(“Video”) and316b(“Audio”) for defining the type of digital media units that are displayed for editing.
• In some implementations, the mobile device application can also provide anoverview bar318. As shown, theoverview bar318 includes a number of respective selectable tiles320 corresponding to each object304 in theediting environment302. In some examples, selecting any one of tiles320 simultaneously selects the corresponding object304 for detailed display in theediting environment302.
• In some examples, users can add representations of additional digital media units (or other content) to theediting environment302. Users can also modify digital media units that are already represented in theediting environment302. For example, and with reference toFIG. 3B, when an existing object (e.g., object304a) is selected and displayed, various selectable editing icons are displayed within theediting environment302. In the depicted embodiment, the editing icons include aninsert icon322a, aninsert icon322band anediting icon324. Selecting either ofinsertion icons322a,322benables the user to insert a digital media unit intoediting environment302 before or after the displayed object, respectively. In some examples, a menu is displayed in response to user selection of aninsertion icon322a,322b, which menu includes object, transition and title options. The user can select object from the menu to insert select a digital media object for insertion. The user can select transition from the menu to select a transition for insertion. The user can select title from the menu to input a title for insertion.
• Selectingediting item324 enables the user to modify the digital media unit represented by the selected object. In some implementations, selectingediting item324 allows the user to access one or more contextual editing options. For example, when a selected object is representative of a video clip, the mobile device application enables users to: add and/or remove frames, adjust a default playback speed, pan and/or zoom (e.g., via the Ken Burns effect or traditional cropping), etc. Similarly, when the selected object is representative of a still image, the mobile device application enables the user to: pan and/or zoom (e.g., via the Ken Burns effect or traditional cropping), add filters, etc.
• FIG. 3C shows a place holder for an additional object326 to be imported intoediting environment302 and inserted betweenobjects304aand304b. For example, the depiction ofFIG. 3C is provided in response to user selection of theinsert icon322bofFIG. 3B, and making a selection from the displayed menu. In the depicted example, the object326 can be representative of a transition effect to be inserted between the digital media units represented by theobjects304aand304b. As noted above, the object326 can be inserted manually by a user utilizingediting items322a,322b. In some examples, insertion of an object can be achieved automatically by the mobile device application. As shown, when the object326 is selected, variousselectable editing icons322a,322b,324 are displayed withinediting environment302. Selecting either ofediting items322a,322benables the user to import additional content (e.g., additional representative objects) intoediting environment302.
• Selectingediting icon324 enables the user to modify a default transition effect represented by object326.FIG. 3D shows a menu ofselectable items332 for indicating a desired type of transition effect (e.g., “Default”, “Crossfade”, “Slide Left”, “Fall”, “Grow”, “Shrink”, etc.). The transition effect provides a visual transition between objects when the video is played. In some implementations, the transitions represented byselectable items332 are organized in a library stored inmemory104 ofmobile processing device100. In some examples, additional transitions can be added to the library (for example, additional transitions can be downloaded from a remote server or a computing system) and represented in the menu by additional selectable items.
• As described above,selectable toggle items316a(“Video”) and316b(“Audio”) can define the type of digital media units that are displayed for editing.FIG. 3E showsediting environment302 whenitem316bis selected. In the depicted example,audio slots328a,328bare provided andrecording slot330 is provided. Theaudio slots328a,328bcan each be populated with an audio object. In the depicted example, theaudio slots328a,328bare populated by audio objects represented byaudio representations338a,338b, respectively, and therecording slot330 is empty. In this example,audio representations338a,338bare provided in the form of linear segments aligned with a video timeline340 (e.g., a frame-by-frame timeline). In some implementations, however, audio representations can be provided in the form of discrete objects that are movable withinediting environment302. In some example, when an audio slot is empty, an insert icon is displayed. In response to user selection of the insert icon, an audio object selection menu can be displayed, which the user can navigate to select an audio object for populating the audio slot.
• Audio editing icons334a,334b,336 are displayed. Theaudio editing icons334a,334b,336 andaudio representations338a,338boverlay frames of the objects that make up the video. Selecting either of editingicons334a,334benables the user to edit an audio track represented by theaudio representation338a,338b, respectively. In some examples, audio editing options include accelerating or decelerating the audio playback speed and/or setting start and end time points for the respective audio object relative to a timeline of the underlying video. Selectingicon336 enables the user to record an audio file and import a representation of the recorded file intoediting environment302.
• As described above, selecting the appropriate menu item enables the user to observe a preliminary version of the merged digital media unit. The merged digital media unit can include a combination of the digital media units and effects embodied by the representations imported intoediting environment302. In some implantations, a number of the digital media units and effects are combined in a chronological order prescribed by the arrangement of discrete objects inediting environment302.FIG. 3F shows a preview screen presented to users viadisplay112 ofmobile processing device100. As shown, the preview screen includes aselectable media item342. Selecting the media item can allow users to observe the merged digital media unit. Additionally, in this example,menu bar306 includes selectable menu item344 (“Export”), in addition to the various other menu items described above. Selectingmenu item344 can allow users to export the merged digital media unit to a specified location inmemory104 ofmobile processing device100.
• Although described in the context of a mobile device application operating on a mobile processing device, it is appreciated that implementations of the present disclosure, and all of the functional operations provided herein, can be realized in any suitable type of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the present disclosure can also be realized as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
• A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
• The processes and logic flows described in this disclosure can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
• Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. Elements of a computer can include a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver, to name just a few. Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
• To provide for interaction with a user, implementations of the present disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
• While this disclosure includes some specifics, these should not be construed as limitations on the scope of the disclosure or of what can be claimed, but rather as descriptions of features of example implementations of the disclosure. Certain features that are described in this disclosure in the context of separate implementations can also be provided in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be provided in multiple implementations separately or in any suitable sub-combination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination.
• Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
• Thus, particular implementations of the present disclosure have been described. Other implementations are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. A number of implementations have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure. For example, various forms of the flows shown above can be used, with steps re-ordered, added, or removed. Accordingly, other implementations are within the scope of the following claims.

Claims (20)

What is claimed is:

1. A computer-implemented method for providing a digital video using a mobile device, the method being executed by one or more processors, the method comprising:

displaying, by the one or more processors, a first digital object representing a first digital video and a second digital object representing a second digital video in linear order within an editing environment provided on a mobile device;

displaying, by the one or more processors, a first insert icon on a first side of the first digital object, the first side of the first digital object being between the first digital object and the second digital object;

in response to receiving first user input corresponding to the first insert icon, displaying, by the one or more processors, a first menu, the first menu including options for selection of different transitions for insertion;

receiving, by the one or more processors, a second user input associated with the first menu, the second user input identifying a selected transition to be inserted to the first side of the first digital object; and

displaying, by the one or more processors, a third digital object representing the selected transition within the editing environment, the third digital object located between the first digital object and the second digital object, the third digital object being different from the first insert icon.

2. The method ofclaim 1, wherein the first insert icon is positioned within the editing environment before the first digital object.

3. The method ofclaim 1, further comprising providing the digital video by merging the first digital video, the second digital video, and the selected transition.

4. The method ofclaim 1, further comprising providing the digital video to a target, the digital video including a video stream obtained by merging the first digital video, the second digital video, and the selected transition.

5. The method ofclaim 4, wherein the target comprises one of a display or a memory of the mobile device.

6. The method ofclaim 1, wherein the first menu comprises a plurality of selection elements, individual selection element corresponding to individual ones of the options for selection of the different transitions.

7. The method ofclaim 6, wherein the second user input comprises a selection of one of the plurality of selection elements.

8. The method ofclaim 1, further comprising:

displaying a second insert icon on a second side of the first digital object;

receiving third user input, the third user input corresponding to the second insert icon; and

in response to receiving the third user input, displaying a second menu.

9. The method ofclaim 1, further comprising displaying an overview timeline, the overview timeline comprising thumbnail representations of each of the first digital object, and the second digital object.

10. The method ofclaim 1, further comprising displaying a graphical representation of an audio object, the graphical representation being aligned with a video timeline of the digital video.

11. The method ofclaim 1, further comprising applying an editing effect to one or more of the first digital video, the second digital video, and the selected transition, the editing effect comprising one or more of a playback speed adjustment, a pan effect, and a zoom effect.

12. The method ofclaim 1, further comprising:

displaying an edit icon between the first side and a second side of the first digital object; and

in response to receiving third user input corresponding to the edit icon, displaying a second menu, the second menu including options for modification of the first digital video represented by the first digital object.

13. A system, comprising:

a computing device; and

a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for editing a digital video using a mobile device, the operations comprising:

displaying a first digital object representing a first digital video and a second digital object representing a second digital video in linear order within an editing environment provided on the mobile device;

displaying a first insert icon on a first side of the first digital object, the first side of the first digital object being between the first digital object and the second digital object;

in response to receiving first user input corresponding to the first insert icon, displaying a first menu, the first menu including options for selection of different transitions for insertion;

receiving a second user input associated with the first menu, the second user input identifying a selected transition to be inserted to the first side of the first digital object; and

displaying a third digital object representing the selected transition within the editing environment, the third digital object located between the first digital object and the second digital object, the third digital object being different from the first insert icon.

14. The system ofclaim 13, wherein the operations further comprise:

displaying an edit icon between the first side and a second side of the first digital object; and

in response to receiving third user input corresponding to the edit icon, displaying a second menu, the second menu including options for modification of the first digital video represented by the first digital object.

15. The system ofclaim 13, wherein the operations further comprise:

providing the digital video by merging the first digital video, the second digital video, and the selected transition.

16. The system ofclaim 13, wherein the first menu comprises a plurality of selection elements, individual selection element corresponding to individual ones of the options for selection of the different transitions.

17. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for editing a digital video using a mobile device, the operations comprising:

displaying a first digital object representing a first digital video and a second digital object representing a second digital video in linear order within an editing environment provided on a mobile device;

displaying a first insert icon on a first side of the first digital object, the first side of the first digital object being between the first digital object and the second digital object;

in response to receiving first user input corresponding to the first insert icon, displaying a first menu, the first menu including options for selection of different transitions for insertion;

receiving a second user input associated with the first menu, the second user input identifying a selected transition to be inserted to the first side of the first digital object; and

displaying a third digital object representing the selected transition within the editing environment, the third digital object located between the first digital object and the second digital object, the third digital object being different from the first insert icon.

18. The non-transitory computer-readable storage medium ofclaim 17, wherein the operations further comprise:

displaying an edit icon between the first side and a second side of the first digital object; and

in response to receiving third user input corresponding to the edit icon, displaying a second menu, the second menu including options for modification of the first digital video represented by the first digital object.

19. The non-transitory computer-readable storage medium ofclaim 17, wherein the operations further comprise:

providing the digital video by merging the first digital video, the second digital video, and the selected transition.

20. The non-transitory computer-readable storage medium ofclaim 17, wherein the first menu comprises a plurality of selection elements, individual selection element corresponding to individual ones of the options for selection of the different transitions.

Images