US20230104738A1 - Apparatus, method, and system for presentation of multimedia content including augmented reality content - Google Patents

Abstract

The present disclosure relates to methods, devices, and systems for presentation of multimedia content. In an aspect of the present disclosure, a method includes receiving, at a processor of a mobile device, a plurality of images from a camera, at least one image of the plurality of images including an object. The method also includes displaying a camera video stream based on the plurality of images, displaying three dimensional (3D) augmented reality content in the camera video content, and displaying video content based on the 3D augmented reality content. The video content is associated with the object.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• The present application is a continuation of U.S. patent application Ser. No. 17/075,159, filed on Oct. 20, 2020, which is a continuation of U.S. patent application Ser. No. 16/404,104, filed on May 6, 2019, which claims benefit of U.S. Provisional Application No. 62/732,938 filed Sep. 18, 2018, and claims the benefit of U.S. Provisional Application No. 62/676,748 filed May 25, 2018, each of which is incorporated by reference herein in its entireties.
TECHNICAL FIELD
• The present disclosure generally relates to a multimedia content delivery and presentation, and, but not by way of limitation, to methods, devices, and systems for delivering or presenting multimedia content including augmented reality content and video content in a camera video stream.
BACKGROUND
• With the proliferation of electronic devices, augmented reality (AR) has been implemented in a variety of contexts. Conventional implementations of augmented reality (AR) include a live direct or indirect view of a physical, real-world environment with elements that are augmented by computer-generated content, such as a graphic (e.g., an animation) and audio.
• AR functionality is typically implemented using mark-less tracking or marker-based tracking. An example of marker-less based tracking is described in U.S. Pat. No. 8,797,353 in which an augmented reality (AR) feature is coupled with a physical object and marker-less tracking is used to present the AR feature on the physical object. For example, a first user captures an image of the physical object using an image capture device, attaches AR content to a location on the physical object represented in the image and sends a rich media message (associated with the AR content attached to the image) to a second user, and transfers the physical object to the second user. In some implementations, the first user may also attach a file, such as a text file (e.g., a text message), music file, or other personal message file. Using an AR viewer and the rich media message, the second user can view the physical object and see, via display device, the AR feature appearing on the physical object. Such distribution of AR content in this manner is not without several limitations that impact use of the AR feature. To illustrate, the first user is required to capture an image of the first object, the first user has to know or have the ability to send the rich media message to the second user, the physical object as to initially be in the possession of the first user and then provided to the second user, and the marker-less tracking if often unreliable, thereby disrupting presentation of the AR feature.
• In marker-based AR, an application (e.g., an AR viewer) is configured to recognize one or more markers present in a real-world environment. In such implementations, the application is programmed to recognize the one or more markers, such as a geospatial marker, a 2D marker, and/or a 3D marker. The AR viewer may detect a marker and launch the AR content based on recognition of the marker. In such systems, a device that includes the AR content is often required to store information for multiple markers and/or a large amounts of data corresponding to AR content for each marker.
• Presenting AR content, such as 3D AR content in a live video stream can be challenging because 3D content often requires multiple models or point clouds, complex localization and mapping function, and is very data and computation intensive. Additionally, difficulty and complexity exists in providing AR content that interacts with real world objects, as well as other media content, in a realistic manner. AR systems are still in their infancy due the technical complexities of implementing such systems. As AR content continues to grow in popularity and use, many challenging aspect of AR functionality will need to be identified and overcome to produce faster and more efficient AR systems.
SUMMARY
• The present disclosure describes devices, systems, methods, and computer-readable storage media for providing augmented reality content in a camera video stream. For example, a mobile device having a camera and a display may display a live video stream captured by the camera and modify the live video stream with augmented reality content and other video content, such as pre-recorded video content that corresponds to an object, such as a packaged delivered to a user.
• In some implementations, devices, systems, methods, and computer-readable storage media of the present disclosure enable a requester of an order (for delivery of an object to a recipient) to request a multimedia message to be made available. The multimedia message may include augmented reality (AR) content that is provided in a live video stream at a device associated with the recipient. In addition, the requester (order entity) may provide video content to be included in the multimedia message such that the video content is presented along with the AR content—e.g., the video content may be present with and included in 3D AR content. To illustrate, video content may be assigned or attached to an order of an item to be delivered to a recipient. For example, the item may be included in a package having a marker and the package may be delivered to the recipient. Additionally, an identifier that corresponds to the video content and that is associated with the package is provided to the recipient.
• In some implementations, the recipient of the package may utilize a mobile device to scan or input the identifier and may send corresponding identifier data to a server associated with the order. Responsive to the identifier data, the mobile device may receive one or more content items, such as the video content, AR content (or an indication of selected AR content), text content, and/or image content. The mobile device (e.g., a processor) may initiate an AR viewer that causes the mobile device to provide at least one live video stream associated with a camera of the mobile device. The mobile device may display a target to direct a user to position the mobile device in a predetermined location with respect to a marker coupled to the package. By providing the target (e.g., a template) for the marker, a user is able to position the mobile device in an initial position, orientation, and distance with respect to the marker. In response to detection of the marker, the mobile device may establish a coordinate system with respect to the marker and initiate a multimedia message in which the AR content (e.g., 3D AR content) and the video content is included in the live video stream from the camera.
• In some implementations, the mobile device includes presentation data associated with a location (e.g., 3D position information) of the video content within the 3D AR content. The presentation data may have an initial value that is calibrated to the coordinate system established based on detection of the marker. After detection of the marker, the mobile device may track one or more characteristics of the marker to determine a change in position (e.g., distance, rotation, translation) of the mobile device with respect to changes in features of the marker (e.g., size, angle, horizontal and/or vertical position within the display). The mobile device may modify the position data based on the determined change in position of the marker and/or the mobile device. In some implementations, the mobile device may also generate a set of predicted presentation data values and select one of the predicted set of values for presentation of the video content. The selected value may be determined based on one or more inputs, such as motion data from a sensor (e.g., an inertial sensor) of the mobile device.
• The devices, systems, methods, and computer-readable storage media of the present disclosure address the problem of complex and processor intensive operations that create unrealistic interactions between AR content and real world objects in a live camera stream. To illustrate, in some implementations, the devices, systems, methods, and computer-readable storage media of the present disclosure provide rules that improve generation and/or presentation of multimedia content that includes a live camera stream, AR content, and pre-recorded video content by permitting automation of image processing of the pre-recorded video content. Additionally, or alternatively, the devices, systems, methods, and computer-readable storage media of the present disclosure provide integration of the pre-recorded video content with the AR content, and integration of the AR content with the live camera stream, to enable a realistic presentation of the AR content and the pre-recorded video content with at least one real world object included in the live camera stream. To illustrate, the present disclosure may utilize a plurality of rules for conditioning for inclusion into the video stream and something for a realistic playback, such as using a reel-to-reel projector or a monitor. Additionally, or alternatively, one or more virtual objects may be used to determine one or more portions of AR content to occlude for presentation of in the live camera stream. The one or more virtual objects may include simple shapes that are easy to track and/or a position at least one of the virtual objects (in relation to a coordinate system or another virtual object) may be dynamically changed based on a determined camera pose.
• Thus, present disclosure describes systems, devices, and methods for providing multimedia content that corresponds to an object, such as a box, vase, platter, toy, etc., delivered to the recipient. In such implementations, a sender/requestor of the order is not required to have access to the package prior to delivery of the package to the recipient. The mobile device associated with the recipient is configured to implement a set rules, such as an AR viewer application, that provided one or more conditions for establishing a coordinate system with respect to the package and/or presenting video content. For example, the AR viewer application may dynamically determine video content placement data based on the coordinate system. To illustrate, the video content data may be determined as a function of marker characteristics and a movement prediction algorithm. Based on the video content placement data, the set of rules are further configured to determine and implement video content presentation characteristics (e.g., orientation, translation, size, scale, perspective, etc.) to dynamically adjust the video content for presentation. Accordingly, the video content may be efficiently presented with a live video stream such that the video content interacts with the AR content included in the live video stream. As another example, pre-recorded video content may be automatically processed and combined with AR content to provide a realistic presentation of the AR content and the processed video content in a live camera stream. The displayed video content may further be presented based on movement and placement of at least a portion of the AR content such that an occlusion algorithm, a collision algorithm, and/or a shadow algorithm further coordinate and/or synchronize movement of the mobile device with presentation of the AR content and the video content within the live video stream. Thus, the present disclosure advantageously provides multimedia content by establishing a coordinate system in a controlled manner such that the one or more rules may be efficiently applied to one or more virtual objects for presentation of AR content and pre-recorded video content in a live camera stream. Additionally, or alternatively, the one or more rules may be applied to dynamically determine adjustments/updates to presentation data associated with the video content and to select the presentation data based at least on features of marker and/or movement or predicted movement of a mobile device.
• In an aspect of the present disclosure, one or more methods comprise: receiving, at a processor of a mobile device, a plurality of images from a camera, at least one image of the plurality of images including an object; displaying a camera video stream based on the plurality of images; displaying three dimensional (3D) augmented reality content in the camera video stream; and displaying video content based on the 3D augmented reality content, the video content associated with the object. In some such methods, the object includes a package having one or more items, the package delivered to a user associated with the mobile device, and presentation of the camera video stream including the 3D augmented reality content and the video content is associated with a multimedia message.
• In some of the foregoing implementations, the one or more methods may comprise: receiving, at the mobile device, an application; storing the application at a memory of the mobile device; and accessing the application. Additionally, or alternatively, the one or more methods may comprise: receiving, at the processor, a first image from the camera, the first image including an identifier associated with the object; detecting the identifier included in the image; and sending, from the mobile device to a server, at least a portion of the image including the identifier or identifier data based on the identifier.
• In some of the foregoing implementations, the one or more methods comprise: receiving, at the mobile device from the server, the video content, where the video content corresponds to the identifier and is associated with a sender of the object, and where the video content comprises audio/visual content. Additionally, or alternatively, the one or more methods may comprise: accessing, by the processor, the camera of the mobile device; and initiating, by the processor, a prompt to capture the image of the identifier, wherein, optionally, the identifier is coupled to the object. In some such implementations, wherein the identifier comprises a marker coupled to the object.
• In some of the foregoing implementations, the one or more methods may comprise: receiving, at the processor, a second image from the camera; and detecting, by the processor based on the second image, the object or a marker coupled to the object. Additionally, or alternatively, the one or more methods may comprise: tracking, by the processor, at least a portion of the marker in the plurality of images; determining position information associated with the marker in one or more images of the plurality of images; and displaying the 3D augmented reality content within the camera video stream based on the position information. In some such implementations, the one or more methods may also comprise: receiving, by the processor, placement data associated with the 3D augmented reality content; and determining, by the processor based on the position information and the placement data indicates, a position of the video content in relation to the 3D augmented reality content and/or one or more presentation characteristics of the video content.
• In some of the foregoing implementations, the one or more methods may comprise: receiving text data, image data, or both at the mobile device from the server based on the identifier; displaying the text data, the image data, or both in the camera video content; and optionally, arranging the text data, the image data, or both based on the 3D augmented reality content. Additionally, or alternatively, the one or more methods may comprise sending, from the mobile device, a notification associated with a completion of the 3D augmented reality content, the video content, or both. In some such implementations, the one or more methods may also comprise: initiating a prompt to send a message based on a completion of the 3D augmented reality content, the video content, or both; receiving an input to send the message; and sending the message to the server, a device associated with a sender of the object, or both.
• In some of the foregoing implementations, the one or more methods comprise: establishing communication with a server; and initiating, via the server, an order for delivery of an item to a recipient. In some such implementations, initiating the order comprises: generating order request information; and communicating the order information to a server. For example, generating the order request information may comprise: selecting the item; and selecting an option to deliver a multimedia message to the recipient. In such implementations, selecting the option to deliver the multimedia message may include: selecting order 3D augmented reality content; providing order video content; providing a text content; and/or providing an image. Additionally, or alternatively, the one or more methods may comprise: receiving confirmation of delivery of the item to the recipient; and/or receiving a notification that the multimedia message has been viewed.
• In another aspect of the present disclosure, one or more devices include: a processor; and a memory storing one or more instructions that, when executed by the processor, cause the processor to: receive a plurality of images from a camera, at least one image of the plurality of images including an object; display a camera video stream based on the plurality of images; display three dimensional (3D) augmented reality content in the camera video stream; and display video content based on the 3D augmented reality content, the video content associated with the object. In some such implementations, the one or more devices may further include: a display coupled to the processor; and/or the camera coupled to the processor. Additionally, the one or more instructions may be included in an application that is stored at the memory, and/or the memory may further be configured to store the 3D augmented reality content, the video content, or a combination thereof.
• In some of the foregoing implementations, the one or more instruction, when executed by the processor, may further cause the processor to initiate storage at the memory of multimedia content presented via the display, the multimedia content associated with the camera video stream, the 3D augmented reality content, the video content, or a combination thereof. Additionally, or alternatively, in some of the foregoing implementations, the one or more devices may include: a network interface coupled to the processor, where the network interface comprises: a transmitter coupled to the processor, the transmitter configured to transmit one or more wireless signals; and/or a receiver coupled to the processor, the receiver configured to receive one or more wireless signals.
• Some implementations of the present computer-readable storage mediums (e.g., computer-readable storage devices) store instructions that, when executed by a processor, cause the one or more processors to perform operations comprising: receiving, at a processor of a mobile device, a plurality of images from a camera, at least one image of the plurality of images including an object; displaying a camera video stream based on the plurality of images; displaying three dimensional (3D) augmented reality content in the camera video stream; and displaying video content based on the 3D augmented reality content, the video content associated with the object.
• In another aspect of the present disclosure, one or more methods comprise: receiving, at a processor of a mobile device, video content including a plurality of frames; displaying, at a display of the mobile device, a camera video stream generated by an image capture device of the mobile device, the camera video stream including an object; displaying a first virtual object in the camera stream, the first virtual object associated with presentation of a version of the video content in the camera video stream; providing a light source associated with the first virtual object, the light source positioned to correspond with presentation of the version of the video content; and presenting the version of the video content based on the first virtual object and concurrently with the virtual light source. In some implementations of the one or more methods, the object comprises a real world object; the first virtual object comprises a film projector or a monitor; and/or the version of the video content comprises the video content or a modified version of the video content. Additionally, or alternatively, the one or more methods may include providing a user interface concurrently with display of the camera stream, the user interface configured to enable recording of the camera stream including the version of the video content.
• In some of the foregoing implementations, the one or more methods may comprise: displaying a second virtual object in the camera stream, where, when the first virtual object comprises the film projector: the virtual light source is positioned at the first virtual object; the second virtual object comprises a projector screen; and the version of the video content is projected from the film projector toward the second virtual object. Alternatively, when the first virtual object comprises the monitor: the virtual light source is positioned within the first virtual object; and the version of the video content is project from within the first virtual object onto a screen of the monitor.
• In some of the foregoing implementations, the one or more methods may comprise: buffering the one or more frames of the video content; and applying a mask to one or more frames of the video content to generate the version of the video content, wherein the mask is configured to blur one or more edges of the one or more frames to create a falloff effect. In some such implementations, the one or more methods may further comprise: detecting a marker or the object in the camera stream; and establishing a coordinate system based on the detected mark or the detected object, the coordinate system associated with a real world environment or a virtual environment. The one or more methods may also include: positioning, based on the coordinate system, a two dimensional (2D) plane in a virtual environment; determining an opening in the 2D plane; determining, based at least on the 2D plane, the opening, or both, a first portion of a third virtual object and a second portion of the third virtual object; and presenting the first portion of the third virtual object in the camera video stream based on the coordinate system. The first portion may comprise an occluded portion of the third virtual object; and the second portion may comprise a non-occluded portion of the third virtual object.
• The one or more methods may also include: positioning, based on the coordinate system and based on the opening, an axis in the virtual environment; determining a camera pose associated with the image capture device; and positioning a fourth virtual object in the virtual environment based on the camera pose, the fourth virtual object positioned adjacent to the 2D plane; wherein the first portion of the third virtual object and the second portion of the third virtual object are further determined based on the fourth virtual object. Additionally, or alternatively, the one or more methods may include positioning, based on the opening, a fifth virtual object in the virtual environment, the fifth virtual object positioned adjacent to the 2D plane; wherein the first portion of the third virtual object and the second portion of the third virtual object are further determined based on the fifth virtual object.
• The foregoing has outlined rather broadly the features and technical advantages of the present disclosure in order that the detailed description that follows may be better understood. Additional features and advantages will be described hereinafter which form the subject of the claims. It should be appreciated by those skilled in the art that the conception and specific examples disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes described herein. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the disclosure as set forth in the appended claims. The aspects which are characterized herein, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present disclosure.
BRIEF DESCRIPTION OF THE FIGURES
• For a more complete understanding of the present example inventions, reference is now made to the following descriptions taken in conjunction with the accompanying figures, in which:
• FIG.1 illustrates an example of a system for providing multimedia content associated with delivery of a package;
• FIG.2 illustrates an example of an order form associated with providing multimedia content associated with delivery of a package;
• FIG.3 illustrates an example of a data structure associated with providing multimedia content associated with delivery of a package;
• FIG.4 illustrates an example of a package and a card associated with providing multimedia content;
• FIG.5 illustrates an example of displayed frames of multimedia content associated with delivery of a package;
• FIG.6 illustrates another example of displayed frames of multimedia content associated with delivery of a package;
• FIG.7 illustrates an example of a frame of multimedia content associated with delivery of a package;
• FIG.8 illustrates another example of a frame of multimedia content associated with delivery of a package;
• FIGS.9A,9B,9C, and9D illustrate examples of frames of multimedia content associated with delivery of a package;
• FIG.10 illustrates another example of displayed frames of multimedia content associated with delivery of a package;
• FIG.11 illustrates an example of content from a mobile device displayed at an electronic device;
• FIG.12 illustrates a flow diagram of an example of a method of operating an electronic device;
• FIG.13 illustrates a flow diagram of another example of a method of operating a server;
• FIG.14 illustrates a flow diagraph of another example of a method of operating a mobile device;
• FIG.15 illustrates an example of displayed frames of multimedia content associated with delivery of a package;
• FIG.16 illustrates an example of displayed frames of multimedia content associated with delivery of a package;
• FIG.17 illustrates an example of a first technique for occlusion of AR content;
• FIG.18 illustrates an example of a second technique for occlusion of AR content; and
• FIG.19 illustrates a flow diagraph of another example of a method of operating a mobile device.
DETAILED DESCRIPTION OF THE INVENTION
• Particular implementations of the present disclosure are described below with reference to the drawings. In the description, common features are designated by common reference numbers throughout the drawings. As used herein, various terminology is for the purpose of describing particular implementations only and is not intended to be limiting of implementations. For example, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It may be further understood that the terms “comprises” and “comprising” may be used interchangeably with “includes” or “including.” Additionally, it will be understood that the term “wherein” may be used interchangeably with “where.”
• As used herein, an ordinal term (e.g., “first,” “second,” “third,” etc.) used to modify an element, such as a structure, a component, an operation, etc., does not by itself indicate any priority or order of the element with respect to another element, but rather merely distinguishes the element from another element having a same name (but for use of the ordinal term). The term “coupled” is defined as connected, although not necessarily directly, and not necessarily mechanically; two items that are “coupled” may be unitary with each other. The phrase “and/or” means and or. To illustrate, A, B, and/or C includes: A alone, B alone, C alone, a combination of A and B, a combination of A and C, a combination of B and C, or a combination of A, B, and C. In other words, “and/or” operates as an inclusive or.
• Further, a device or system that is configured in a certain way is configured in at least that way, but it can also be configured in other ways than those specifically described. The feature or features of one embodiment may be applied to other embodiments, even though not described or illustrated, unless expressly prohibited by this disclosure or the nature of a described implementation.
• Referring toFIG.1, an example of asystem100 for delivering multimedia content associated with delivery of a package is shown.System100 includes anelectronic device110, one or more servers, such as arepresentative server130, amobile device150, and adistribution entity180, each of which may be coupled via anetwork190.Network190 may include a wireless network, a wired network, or a combination thereof. For example,network190 may include one or more of a local area network (LAN), metropolitan area network (MAN), wide area network (WAN), wireless LAN (WLAN), the Internet, intranet, extranet, cable transmission system, cellular communication network, and/or the like.
• Electronic device110 may include anetwork interface112, one ormore processors114, amemory116, and acamera120.Network interface112 may be configured to be communicatively coupled to one or more external devices, such asserver130 and/ormobile device150, via one or more networks (e.g.,190). For example,network interface112 may include a transmitter, a receiver, or a combination thereof (e.g., a transceiver).Memory116 may storeinstructions118, that when executed by the one ormore processors114, cause the processor(s)114 to perform operations as described further here. For example, the one ormore processors114 may perform operations as described with reference toFIG.12. In some implementations,memory116 may store an application (e.g., a software program) associated with ordering and/or delivery an item. As illustrative, non-limiting examples, the item may include food (e.g., a backed good, fruit, vegetables, meat, processed food, etc.), a plant (e.g., flowers, seeds, etc.), clothing (e.g., shoes, a shirt, a hat, etc.), another item (e.g., a gift basket, a bouquet of flowers in a vase, an arrangement of food, etc.), or a combination thereof.Camera120 may include an image capture device configured to generate image data, such as image data and/or video data. In some implementations,electronic device110 may include one or more input/output (I/O) devices. The one or more I/O devices may include a mouse, a keyboard, a display device, thecamera120, a microphone, other I/O devices, or a combination thereof.
• Electronic device110 may include or correspond a communications device, a mobile phone, a cellular phone, a satellite phone, a computer, a tablet, a portable computer, a display device, a media player, or a desktop computer. Additionally, or alternatively, theelectronic device110 may include a set top box, an entertainment unit, a navigation device, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a video player, a digital video player, a digital video disc (DVD) player, a portable digital video player, a satellite, a vehicle, any other device that includes a processor or that stores or retrieves data or computer instructions, or a combination thereof.
• Server130 may include anetwork interface132, one ormore processors134, and amemory136.Network interface132 may be configured to be communicatively coupled to one or more external devices, such aselectronic device110,mobile device150, and/or a device ofdistribution entity180, via one or more networks (e.g.,190). For example,network interface132 may include a transmitter, a receiver, or a combination thereof (e.g., a transceiver).Memory136 may storeinstructions138,preview content139,order data140.Instructions138 may be configured to, when executed by the one ormore processors134, cause the processor(s)134 to perform operations as described further here. For example, the one ormore processors134 may perform operations as described with reference toFIG.13. In some implementations,memory136 may store an application (e.g., a software program) associated with ordering an item for delivery.Preview content139 may include pre-determined content for generating a preview of multimedia content (e.g., multimedia message preview194). To illustrate, preview content may include pre-determined video content of a representative item (e.g., a package), and augmented reality content and/orvideo content193 may be combined withpreview content139 to generatemultimedia message preview194. In some implementations,server130 may providemultimedia message preview194 toelectronic device110.
• Theorder data140 may include a data structure, such as a table, that includes order information (e.g.,192) for one or more orders. For example, each order may be arranged in theorder data140 as anentry142. To illustrate,entry142 may include at least a portion oforder information192. An illustrative example of order data140 (e.g., entry142) is described further herein with reference toFIG.3. Additionally, or alternatively,memory136 may be configured to store additional data, such as content associated with a multimedia message as described further herein. For example, such content may includevideo content193, AR content, text data, image data (e.g., a logo, picture, graphic, etc.), or a combination thereof.
• In some implementations,server130 may comprise a general purpose processor-based system (e.g., PC and/or server system having a processor, memory, suitable I/O functionality, and OS) operating under control of an instruction set to interface withelectronic device110,mobile device150, and/ordistribution entity180. In some implementations,server130 may include one or more input/output (I/O) devices. The one or more I/O devices may include a mouse, a keyboard, a display device, thecamera120, other I/O devices, or a combination thereof. Additionally, or alternatively, in some implementations,system100 may include multiple servers. To illustrate, one or more features or operations described with reference toserver130 may be included in or performed by another server. For example, in some implementations,video content193 generated byelectronic device110 may be stored at a server that is different fromserver130, and order information192 (that includes address information corresponding tovideo content193 stored at the other server) may be received atserver130. In a particular implementation, the other server includes a cloud server.
• Mobile device150 may include anetwork interface152, one ormore processors154, abuffer155, amemory156, acamera168, and adisplay172.Network interface152 may be configured to be communicatively coupled to one or more external devices, such aselectronic device110 and/orserver130, via one or more networks (e.g.,190). For example,network interface152 may include a transmitter, a receiver, or a combination thereof (e.g., a transceiver).Memory136 may storeinstructions158 andapplication160.Instructions158 may be configured to, when executed by the one ormore processors154, cause the processor(s)154 to perform operations as described further herein. For example, the one ormore processors154 may perform operations as described with reference toFIG.14 and/or19. In some implementations, the one ormore processors154 may include multiple processors, such as a first processor to process image data from a first camera (e.g., a forward facing camera) and a second processor to process image data from a second camera (e.g., a rear facing camera).
• Application160 (e.g., a software program) may be associated with ordering and/or delivering an item. In some implementations,application160 corresponds to an AR viewer and includes one or more sets of rules or instructions to perform operations as described herein for presenting multimedia content that includesAR content162 included in a camera video stream (associated with images170) and in whichvideo content193 is dynamically presented based on a position ofcamera168 and further based on the AR content within the camera video stream. For example, the one or more sets of rules may direct the recipient to position a camera with respect topackage186 and may establish a coordinate system based on detection of a marker.Application160 may include augmented reality (AR)content162, amask163,placement data163, or a combination thereof.Mask163 is configured to be used to modifyvideo content193 to generate modified video content. Theplacement data163 may be used for presenting/positioning AR content162 and/orvideo content193 withinAR content162. Additionally,placement data163 may be used to modify characteristics, such as size and/or scale, of thevideo content193.Camera168 may include an image capture device configured to generate an image170 (e.g., image data). For example,image170 may include image data and/or video data. In some implementations,electronic device110 may include one or more input/output (I/O) devices. The one or more I/O devices may include a mouse, a keyboard, a display device, thecamera120, a microphone, a speaker, other I/O devices, or a combination thereof.
• In some implementations,AR content162 includes one or more AR objects, such as a character or avatar, a film projector, a screen, a ladder, a saw, a monitor, etc., to be included in a live video stream. Additionally, or alternatively,AR content162 may include one or more virtual objects or models to enable the one or more AR objects to be placed within the live video stream and/or interact with real world objects in the live video stream. For example, the one or more virtual objects or models may be used for collision, occlusion, or both. As illustrative, non-limiting examples, the one or virtual objects or models may include a 2D plane, a line segment (e.g., an axis), a cylinder, a 3D model, or a combination thereof. Examples of techniques for occlusion are described further herein at least with reference toFIGS.17-18. TheAR content162 may also include one or more light sources, one or more sound sources, or a combination thereof, associated with presentation of theAR content162. In some implementations,AR content162 may includemask163.
• Placement data164 may be associated with a location (e.g., 3D position information) ofvideo content193 within the3D AR content162 and/or a projection source ofvideo content193. For example, theplacement data164 may indicate or define a perimeter or boundary associated with presentation ofvideo content193. Theplacement data164 may have an initial value that is calibrated to the coordinate system established based on detection of themarker188. Theplacement data164 may be utilized to displayvideo content193 for a time period during presentation of the multimedia message. For example, presentation of thevideo content193 may be synchronized with theAR content162. In other implementations,placement data164 corresponds to placement ofAR content162. Additionally, or alternatively,placement data164 may be associated with positioning of one or more light sources, one or more audio sources, one or more virtual objects (e.g., a 2D virtual plane) that are not displayed, or a combination thereof.
• Buffer155 is configured to storevideo content193 prior to display of thevideo content193. In some implementations, the stored video content may be modified to generate modified video content. For example,video content193 stored atbuffer155 may be modified based onmask163 to generate a modified version ofvideo content193. It is to be understood that, as described herein, presentation ofvideo content193 viadisplay172 may include presentation of the modified version ofvideo content193.
• Distribution entity180 is configured to provide a system by which orders may be fulfilled and delivered. In some implementations,distribution entity180 may include or correspond to a provider, an acquirer of goods and/or services, a distributor, and/or a shipper. For example,distribution entity180 may be configured to package one or more items to be delivered to a recipient, and to initiate and/or deliver the package. In a particular implementation,server130 corresponds to a provider (e.g., seller) of one or more items anddistribution entity180 corresponds to a shipping entity that includes and inventory of the one or more items and shipping media (e.g., boxes, envelopes, etc.). In such implementations,distribution entity180 may deliver a package (e.g.,186) including one or more items or may operate with a third party for delivery of the package. Thepackage186 may include one or more items, a packaging material (e.g., a box, a crate, a bag, a wrapper, etc.), or a combination thereof. Although described as being separate fromserver130, in other implementations, distribution entity may includeserver130.
• Distribution entity180 may include an electronic terminal182. Electronic terminal may comprise a general purpose processor-based system (e.g., PC and/or server system having a processor, memory, suitable I/O functionality, and OS) operating under control of an instruction set to interface withelectronic device110,server130, and/ormobile device150. Electronic terminal182 may include or correspond a communications device, a mobile phone, a cellular phone, a satellite phone, a computer, a tablet, a portable computer, a display device, a media player, or a desktop computer. Additionally, or alternatively, the electronic terminal182 may include a set top box, an entertainment unit, a navigation device, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a video player, a digital video player, a digital video disc (DVD) player, a portable digital video player, a satellite, a vehicle, any other device that includes
• During operation ofsystem100,electronic device110 may communicate withserver130 to initiate an order for delivery of an item. For example,electronic device110 may access an order form as described with reference toFIG.2. In such implementations,electronic device110 may be associated with or correspond to a sender and/or requestor. In some implementations, the order form may be accessible via a web page or an application (e.g.,160), such as an application stored atmemory116. The application (e.g., a software component) may be associated with a provider, distributor, or shipper of the item. Additionally, or alternatively, the application may be associated with a third party that provides a platform or interface for the provider, distributor, or shipper. As an illustrative, non-limiting example, the application may be associated with a business that delivers items, such as plants (e.g., flowers), food (e.g., cookies, fruit, meat, etc.), clothing, or the like, as illustrative, non-limiting examples. In some implementations, the application may include a social medial/social networking feature, such that the sender/requester can communicate with the recipient via the application.
• Electronic device110 may generateorder information192 to initiate the order. For example,electronic device110 may generateorder information192 to populate the order form.Order information192 may include an indication of the item to be delivered, sender data (e.g., name, address, phone number, email address, payment information, etc.), recipient information (e.g., name, address, phone number, email address, etc.), a message, or a combination thereof.
• Additionally, or alternatively,order information192 may include or indicate that a multimedia message is to be made available to a recipient of the order. The multimedia message may be associated with multimedia content that includes AR content and/or video content included in a camera video stream. In such implementations,order information192 may includevideo content193. As used herein,video content193 includes video data (e.g., a plurality of images), or video data and audio data. In some implementations, the sender/requestor may generatevideo content193 via electronic device110 (e.g.,120) as part of filling out the order form. For example, the order form may be configured to prompt the sender/requestor to generate or provide thevideo content193. Although described as having the video content generated and/or provided with theorder information192 from theelectronic device110, in other implementations,video content193 may be generated and or provided byserver130, distribution entity (e.g., electronic terminal182), or another device or entity. To illustrate, whenserver130 and/ordistribution entity180 correspond to a provider, the provider may include thevideo content193 into theorder information192 to be used in a multimedia message. For example, if the sender/requestor does not request a multimedia message (that includes a personalized message for the recipient) with the order, the provider, distributor, and/or shipper may provide thevideo content193. Additionally, or alternatively,order information192 may include a time value, such as a date, a time, a duration, etc., that corresponds to or defines a time period during which the multimedia content is available. To illustrate, the time period may start or be based on an order time when the order is submitted or a delivery time corresponding to delivery of a package (e.g.,186) to a recipient, as illustrative, non-limiting examples. In other implementations, the multimedia content may be available for a predetermined time period (e.g., a pre-set time period).
• Server130 may receive theorder information192 and may generateentry142 that corresponds to theorder information192. In some implementations, theentry142 may include an order number and/or identifier (e.g.,187). The order number and/or the identifier may be configured to be used as an index to identifyentry142 within theorder data140, to identify storedvideo content193, or both. In a particular implementation, theidentifier187 includes or is associated with a multi-digit alpha/numeric order code that is distinct from the order number. In such implementations, the order code is more complex than the order number and/or is a randomly generated alpha/numeric value, thereby providing a security features for accessing the video content.
• As explained above,server130 may also be configured to identify andstore video content193. For example,video content193 may be stored atmemory136 and may be included inentry142, atmemory136 at a location pointed to byentry142, or at a location remote fromserver130 and identified by (e.g., pointed to)entry142. Although theorder data140 is described as being stored atserver130, in other implementations,order data140 may be stored at a remote location fromserver130, such as at a remote database, at distribution entity180 (e.g., electronic terminal182), or at another location.
• After receivingorder information192,server130 may send (e.g., via a push or a pull operation) anorder request195 to distribution entity180 (e.g., electronic terminal182). Theorder request195 may include at least a portion of theorder information192 and/or at least a portion ofentry142. Thedistribution entity180 may process theorder request195 to create apackage186 that includes one or more items.Package186 may include or correspond to the item or a shipping media, such as a box, a bag, a crate, a vase, etc. In some implementations,package186 may include or correspond to a package identifier (e.g., a package reference) that indicates characteristics of the package, such as dimensions, volume, weight restriction, etc. In such implementations,distribution entity180 may communicate the package identifier and/or one or more characteristics ofpackage186 toserver130 andserver130 may optionally include such information inentry142.
• Thepackage186 may include or be coupled toidentifier187, a representation ofidentifier187, and/or amarker188. To illustrate,identifier187 may be represented as a bar code, a QC code, alphanumeric text, or a combination thereof, as illustrative, non-limiting examples. In some implementations, theidentifier187 may include or be the same as themarker188. In other implementations, themarker188 may include an image, a logo, one or more points or features, etc. Theidentifier187 and/or themarker188 may printed on thepackage186 and/or item, printed on a card or sticker that is coupled to the package and/or item, as illustrative, non-limiting examples. In other implementations, theidentifier187 may include a digital or graphical representation that is communicated to a recipient or to a device (e.g., mobile device150) associated with the recipient.
• Distribution entity180 may initiate physical delivery ofpackage186 to recipient. For example, the recipient may receive adelivery including package186, as described with reference toFIG.4. Based on delivery ofpackage186 to recipient, distribution entity180 (e.g., electronic terminal182) may send adelivery confirmation196 toserver130,electronic device110, and/or another device, such as another device associated with the sender. In some implementations,server130 may receivedelivery confirmation196 and sent a confirmation message (based on delivery confirmation196) to a device (e.g.,110) associated with sender.
• The recipient may be associate with (e.g., be a user of)mobile device150. Based on receivingpackage186, recipient may operatemobile device150 to accessapplication160. In some implementations,application160 includes an AR viewer. Ifmobile device150 does not includeapplication160,mobile device150 may downloadapplication160, such as fromserver130 or another source.Processor154 may execute one or more instructions ofapplication160 to perform operations described herein. To illustrate,processor154 may accesscamera168 and provide one or more images captured bycamera168 viadisplay172. For example, the one or more images (e.g.,170), such as a live video stream, may be provided to thedisplay172 and may be presented in real-time or near real-time. The displayed images presented in real-time or near real-time may be referred to as a camera video stream or a live video stream.
• Processor154 (e.g., application160) may provide a prompt to capture an image ofidentifier187 and/ormarker188, and/or to enter or select an input that corresponds toidentifier187. Processor154 (e.g., application160) may detectidentifier187 and determineidentifier data197 based on the detectedidentifier187. For example,identifier data197 may include a portion of image data. As another example, such as whenidentifier187 includes a barcode, application may process barcode to determine a data value (e.g.,198).Mobile device150 may communicateidentifier data197 toserver130. Responsive to receiving identifier data197 (e.g., identifier187),server130 may accessentry142 to identifyvideo content193 to be provided tomobile device150 and may communicate the video content tomobile device150. Additionally, or alternatively, sever130 may determine whethervideo content193 is available/valid or expired based on the identifier data197 (e.g., the identifier187). In some implementations,server130 may also communicate other information and/or data tomobile device150 based on the receivedidentifier data197. For example,server130 may communicate text data, image data, AR content (or an identifier thereof) tomobile device150 for use in providing multimedia content.
• Mobile device150 may receivevideo content193 from server130 (or from another server). In some implementations,video content193 is streamed from server130 (or from another server) and is received atmobile device150 vianetwork interface152.Video content193 received atmobile device150 may be stored inbuffer155 for presentation viadisplay172 as part of the multimedia content. In some implementations,video content193 stored inbuffer155 may be modified (e.g., using mask163) to generate a modified version ofvideo content193 for presentation viadisplay172. In other implementations,video content193 may be modified at a server (e.g.,130) and the modified version of thevideo content193 may be provided themobile device150.
• After transmission of theidentifier data197, processor154 (e.g., application160) may generate a prompt viadisplay172 for recipient to positioncamera168 so marker188 (e.g., package186) is presented indisplay172. For example,processor154 initiates a target within display as described with reference toFIG.5. By providing the target (e.g., a template) for themarker188, a user is able to position themobile device150 in an initial position, orientation, and distance with respect to themarker188. The initial positioning of themobile device150 with respect tomarker188 may be known by themobile device150. For example,application160 may include initial positioning data, such as dimension ofmarker188, a distance betweenmarker188 andmobile device150, an orientation and/or translation ofmarker188 with reference to a coordinate system implementation responsive to marker detection, etc.
• Processor154 may detectmarker188 and, in response to detection of themarker188, may implement a coordinate system with respect to the marker and initiate presentation of multimedia content including theAR content162 andvideo content193 in at least a portion of the video content stream. In some implementations,processor154 may present theAR content162 and/or thevideo content193 based on theplacement data164.Processor154 may displayAR content162 andvideo content193 within camera video stream. In some implementations, thevideo content193 is displayed in relation to and/or based on a portion of the AR content. Examples of multimedia content are described herein at least with reference toFIGS.5-10 and15-16. Additionally, or alternatively,processor154 may position one or more (virtual) light sources and/or one or more (virtual) audio sources. To illustrate, the one or more virtual light sources may include a point light, a spotlight, or both, as illustrative, non-limiting examples. The one or more virtual audio sources may correspond to audio for theAR content162 and/or audio for thevideo content193. In a particular implementation, one or more virtual audio sources are used to enable the audio content to be included in recording of the multimedia content. In other implementation, processor154 (e.g., application160) may provide at least a portion of the audio directly to a speaker without using a virtual audio source.
• After detection of themarker188,mobile device150 may be configured to track one or more characteristics of themarker188 to determine a change in position (e.g., distance, rotation, translation) ofmobile device150 with respect to changes in features of the marker188 (e.g., changes in size, angle, horizontal and/or vertical position within the display).Mobile device150 may modify/update placement data163 based on the determined change in position of themarker188. In some implementations,mobile device150 may also generate a set of predicted presentation data values and select one of the predicted set of values to be used for presentation ofvideo content193. The selected value(s) may be determined based on one or more inputs, such as motion data from a sensor (e.g., an inertial sensor) ofmobile device150.
• In some implementations,processor154 may access a second camera ofmobile device150 to record a reaction of the recipient during presentation of the multimedia message. In such implementations,processor154 may display a camera video stream corresponding to the second camera as described with reference toFIG.10. In some such implementations,processor154 includes a first processor to process first image data from a first camera (e.g., a forward facing camera) and a second processor to process second image data from a second camera (e.g., a rear facing camera). The first processor, the second processor, or another processor may be configured to combine and/or initiate presentation of the processed first image data, the processed second image data, or a combination thereof.
• Based on presentation of the multimedia content, mobile device150 (e.g., processor154) may sendnotification198 that indicates presentation of the multimedia content. For example,mobile device150 may sendnotification198 toserver130 in response to initiation of the multimedia content, during the multimedia content, or in response to an end of the multimedia content. Based onnotification198,server130 may notify the sender/requester (via electronic device110) that the multimedia content has been provided to recipient.
• In some implementations, after presentation of the multimedia content,processor154 may provide a prompt, via display, to send one or more recordings, such as a recording of the multimedia content presented viadisplay172, send a second recording (e.g., recipient's reaction) associated with a second camera, and/or to record and send a reply message to sender. For example,processor154 may provide a prompt as described with reference toFIG.10. In response to a request to send the one or more recordings,processor154 may sendmobile device content199 toserver130.Server130 may store the one or more recordings which may be accessible toelectronic device110 via an application stored atmemory116. Additionally, or alternatively, the one or more recordings may be stored at another server and a location of the one or more recordings may be provided to theserver130.Electronic device110 may play the one or more recordings. An example of the one or more recordings played viaelectronic device110 is described with reference toFIG.11.
• In some implementations,electronic device110 may provide an image (e.g., a picture of the sender, a logo, a graphic, etc.), audio data, or a combination thereof. In such implementations, electronic device may not providevideo content193 as part oforder information192. When the image, audio data, or both is provided as part of order information, the image, audio data, or both may be provided tomobile device150 and presented withAR content162. For example, the image may be presented inAR content162 as described with reference to thevideo content193. To illustrate, the image may be repeatedly presented instead place ofvideo content193. Additionally, or alternatively, if novideo content193 or image are provided withorder information192, a default image may be displayed, such as a default image provided by a seller of the object, a distributor of the object, etc. Audio data may be provided as described with reference to audio of thevideo content193. When no audio data is included in order information, either as part of thevideo content193 or separate from thevideo content193, default audio content may be presented. For example, the default audio content may be provided by a seller of the object, a distributor of the object, etc.
• In some implementations, the multimedia content may presentvideo content193 such that thevideo content193 appears to be provided byAR content162, such as a film projector (e.g., a reel-to-reel projector) that projects thevideo content193 onto a screen. In such implementations, theapplication160 is configured to provide theAR content162, such that the film projector and the screen are set up within the camera video stream. Thevideo content193 is received bymobile device150 and loaded intobuffer155. Themask163 is applied to thevideo content193, such as to each frame of thevideo content193 to blur the video content at the edges of each frames. Blurring thevideo content193 creates a falloff effect when the video content is provided from the film projector onto a surface.Application160 is configured to project the video content193 (aftermask163 is applied) outward from the film projector onto one or more objects, such as 3D objects, positioned in front of the film projector. For example, thevideo content193 may be projected onto the screen and/or onto a character positioned between the film projector and the screen, as described further herein with reference toFIG.9D.
• Theapplication160 is also configured to position a light source at the film projector object. The light source may include a point light or a spotlight. In some implementations, the light source may be configured in an on state or may be configured to blink. The light source may cause shadows to be cast by one or more objects (e.g., the AR content162), such as a character.
• Theapplication160 is also configured to position one or more audio sources within the multimedia content. For example, an audio source may be positioned at a particular location of a coordinate system established byapplication160. In a particular implementation, an audio source is positioned at an origin of the coordinate system. The audio source may correspond to audio content of theAR content162, audio content of thevideo content193, or both. To illustrate, the audio source may be configured to provide audio content that is synchronized with presentation of theAR content162, thevideo content193, or both.
• In another implementation, theapplication160 may be configured to present the video content193 (or a modified version of the video content193) via a screen, such as a screen of an object representing a monitor or television. In such implementations, the video content may be projected from within the object onto the screen of the object. In such implementations, a light source may be positioned within the object and may radiate outward from the screen of the object.
• In a particular implementation,application160 may include a component, such as an instruction (e.g., a single instruction) defined to enable presentation of thevideo content193 from a film projector (e.g., a reel-to-reel projector) or to enable presentation of thevideo content193 via a monitor. To illustrate, an instruction for a film projector may include one or more inputs (e.g., operands), such as a video content location, a projector output location, and a projector output direction. Based on the inputs of the instruction for the film projector,application160 may provide thevideo content193 to buffer155, applymask163 to generate a modified version of thevideo content193, position a projector source and light source within the camera video content, and provide the modified version of thevideo content193 in the projector output direction. In some implementations, the light source may include a spotlight directed in the projector output direction. In implementations where the instruction is for presentation via a monitor, the instruction may include one or more inputs, such as a video content location, a monitor screen size, and a monitor screen position. Based on the inputs of the instruction for the monitor, application may provide thevideo content193 to buffer155, applymask163 to generate a modified version of thevideo content193, position a projector source and light source within the camera video content, and provide the modified version of thevideo content193 onto a monitor screen as specified. In some implementation, the light source may include a light that radiates outward from the screen. Thus,application160 provides one or more instructions to create realistic representations of AR content that provides video content via a film projector or a monitor.
• One advantage provided by thesystem100 is that multimedia content (e.g.,AR content162 and/or video content193) that corresponds to an object, such aspackage186, may be made accessible to and/or delivered to the recipient.Mobile device150 associated with the recipient is configured to implement a set rules (e.g.,160), such as an AR viewer application, that provided one or more conditions for establishing a coordinate system with respect to thepackage186 and/or for dynamically determining video content placement data (e.g.,193) based on the coordinate system. To illustrate, the video content data for display may be determined as a function of marker characteristics and a movement prediction algorithm. Based on the video content placement data (e.g.,163), the set of rules are further configured to determine and implement video content presentation characteristics (e.g., orientation, translation, size, scale, perspective, etc.) to dynamically adjust thevideo content193 for presentation. Accordingly, thevideo content193 may be efficiently presented with a live video stream (e.g.,170) such that thevideo content193 interacts with theAR content162 included in the live video stream. The displayed video content may further be presented based on movement and placement of at least a portion of theAR content162 such that an occlusion algorithm and/or a collision algorithm further coordinate and/or synchronize movement of themobile device150 with presentation of theAR content162 and thevideo content193 within the live video stream (e.g.,170).
• Other advantages provided by thesystem100 include realistic presentation of video content (e.g.,193) by theapplication160 using theAR content162. For example, theapplication160 may be configured to process thevideo content193 and present the video processed video content within the live video stream to appear as though the video content is provided via a film projector or via a monitor. In such implementations, theapplication160 may be configured to provide the video content and placement of a light source to create a realistic effect. The present disclosure further provides multimedia content by establishing a coordinate system in a controlled manner such that the one or more rules may be efficiently applied to dynamically determine adjustments/updates to presentation data associated with thevideo content193 and to select theplacement data163 based at least on features ofmarker188 and/or movement or predicted movement ofmobile device150.
• Referring toFIG.2, an example of anorder form210 is shown.Order form210 includes one ormore categories220, such as categories222-238, and one or morecorresponding fields240, such as fields242-258.Fields240 may be populated via user input and may be configured to receive text data, an indication of a file location, an input via a selectable option of the field, etc.
• The one ormore categories220 may include asender222, an item(s)224, arecipient226, acontact228, anaddress230,AR content232,video content254,message236, and anaccess code238.Sender field242 may be populated with a name, user name, account number, address, contact information, etc., associated with an individual or entity. For example, the individual or entity may be associated with populating and/or submittingorder form210. Item(s)field244 may be populated with one or more items to be delivered to an recipient, such as a user ofmobile device150. As illustrative, non-limiting examples, the one or more items may include a consumable good (e.g., cookies, fruit, a sandwich, a beverage, alcohol, etc.), clothing (e.g., shoes, a shirt, a hat, etc.), a promotional item (e.g., a poster, a book, a pamphlet, etc.), and/or another item.Recipient field246 may be populated with a name, user name, account number, etc., associated with an individual or entity to receive the one or more items identified in item(s)field244.Contact field248 may be populated with a phone number, email address, etc., associated with the recipient.Address field250 may be populated with a physical address, location description, directions, passcode, etc., associated with delivering the one or more items to the recipient.
• AR content field252 may be populated to indicate AR content is to be provided in a multimedia message that is made available to and/or presented to at least the recipient. In some implementations,AR content field252 may include one or more selectable options of AR content (e.g.,162). To illustrate, a first option may correspond to a first object (e.g., a first animated character), a second option may correspond to a second object (e.g., a second animated character), and a third option may correspond to a holiday theme, as illustrative, non-limiting examples.Video content field254 may be populated to include or identify video content (e.g.,193) to be included in the multimedia message. To illustrate,video content field254 may provide one or more options for the sender to record or attach video content (e.g., a video message). In some implementations, video content provided via order form is communicated to a storage server and a location ID (e.g., a storage address) of the video content at the storage server is populatedvideo content field254. In some such implementations, the storage address populated invideo content field254 may be selectable to cause playback of the video content stored at the storage address. To illustrate, selection of the storage address may initiate presentation ofpreview content139.
• Message field256 may be populated with text data, an image (e.g., a logo, picture, graphic, etc.), or a combination thereof.Access code field258 may be populated with security data, such as a passcode (e.g., password), security phase and/or security answer, etc. Theaccess code field258 may be populated with the security data to limit or restrict access to the multimedia content. To illustrate, the security data may be known by the recipient, or may be provided to the recipient, and may be required to be provided in order for the recipient to receive a portion or an entirety of the multimedia message, such as the video content (e.g.,193) and/or the message (associated with message field256). For example, if the security data is not known by the recipient, the security data may be provided to the recipient, such as via text message, phone message, email, etc. In some implementations, the security data may be provided to the recipient based on delivery of the one or more items (e.g., package186). Referring toFIG.1,server130 may be configured to communicate security data tomobile device150 in response to receivingdelivery confirmation196. Additionally, or alternatively,access code field258 may provide options to limit or restrict a number of times the multimedia message can be viewed, whether the multimedia message is recordable, how long the multimedia message is available/accessible, etc.
• Althoughorder form210 has been described as including particular fields (e.g.,242-258), such fields and descriptions are for illustration purposes and are not intended to be limiting. For example,order form210 may include more fields or fewer fields than described. To illustrate, in some implementations,order form210 further includes a delivery time field associated with a requested time of delivery (e.g., date, time, etc.) of the one or more items to the recipient. Additionally, or alternatively, one or more of fields242-258 may include multiple subfields.
• Thus,FIG.2 illustrates an example of anorder form210 to enable able a sender to initiate a multimedia message to be made available to a receiver of one or more items (e.g., package186). For example,order form210 is configured to enable a sender (e.g., a requester) to provide video content (e.g.,193) to be incorporated into the multimedia message and displayed in relation to and/or with AR content. Additionally,order form210 beneficially enables a security feature to be provided with the multimedia message to restrict or limit access to and/or dissemination of the multimedia message.
• Referring toFIG.3, an example oforder data140 is shown.Order data140 may be generated by, stored at, and/or maintained by one or more servers, such asserver130.Order data140 includes one or more entries, such asentry142.Entry142 includes an one or more fields. The one or more fields my include anorder number310, an item(s)ID312,sender information314,recipient information316, anAR content ID318, avideo content ID320, amessage ID322, and an identifier field324 (corresponding to identifier187). Item(s)ID312 may include or correspond to the one or more items identified and/or described with reference to item(s)field244.Sender information314 may include or correspond to information included insender field242.Recipient information316 may include or correspond to information included inrecipient field246,contact field248, and/oraddress field250.
• AR content ID318 may include AR content (e.g.,162), an AR content identifier (of a plurality of AR content identifiers) and/or an indication of a storage location of AR content. For example, a first AR content identifier may correspond to a first object, a second AR content option may correspond to a second object, a third AR content identifier may correspond to a birthday theme, and a fourth AR content identifier may correspond to a get well theme, as illustrative, non-limiting examples.Video content ID320 may include video data (e.g., video content193) or an indication of a storage location of the video data. For example, the video data may include or correspond to data included invideo content field254.Message ID322 may include message data or an indication of a storage location of the message data. For example, the message data may include or correspond to data included inmessage field256.
• Althoughentry142 is described as including each of the fields310-324, in other implementations,entry142 may include fewer fields or more fields. To illustrate, in a particular implementation,entry142 may not includeAR content ID318 and/or may include an access code field associated withaccess code field258. Additionally, or alternatively, one or more of fields310-324 may include multiple subfields.
• Thus,FIG.3 illustrates an example oforder data140 that is associated with delivery of a package (e.g.,186) and a corresponding multimedia message. For example,order data140 may enable a server to provideorder request195 todistribution entity180 and to provideAR content162 and/orvideo content193 tomobile device150. Additionally, in some implementations, order data140 (e.g., entry142) enablesserver130 to provide information, such asdelivery confirmation196 and/ormobile device content199 toelectronic device110 associated with the sender.
• Referring toFIG.4, an example of adelivery400 is shown.Delivery400 includespackage186 and, optionally, a card410 (e.g., an insert).Package186,card410, or both may includeidentifier187, such as a barcode, a QR code, text, or a combination thereof, as illustrative, non-limiting examples. As shown,card410 includes atext portion412.Text portion412 may include instructions directing the recipient ofpackage186 on how to receive (e.g., access) the multimedia content. For example,text portion412 may direct the recipient to download and/oraccess application160. Usingapplication160, recipient may scan (using camera168)identifier187. Additionally, or alternatively,text portion412 may provide instructions for the recipient to access preview content (e.g., multimedia message preview194) viaserver130. For example, in situations where the recipient does not have access to a mobile device and/or does not have the ability to accessapplication160 or a camera (e.g.,168), thetext portion412 may provide instructions for the recipient to access, such as via a website,multimedia message preview194. Thus,FIG.4 provides an illustrative example ofdelivery400 that includesmarker188 and/oridentifier187 to a recipient to enable presentation of multimedia content as described herein.
• Referring toFIGS.5-10 and15-16, examples of displayed frames of multimedia content associated with delivery of a package (e.g.,168) are shown. For example, one or more of the frames may include or correspond to at least a portion of a camera video stream (e.g.,170) into which AR content (e.g.,162) and video content (e.g.,193) is added. To illustrate, the camera video stream may include or correspond to one ormore images170, and the one or more frames may be displayed via a display, such asdisplay172, of a device. The device may includeelectronic device110 ormobile device150, as illustrative, non-limiting examples. To illustrate, one or more frames ofFIGS.5-10 and15-16 may be generated by an application, such asapplication160.
• As described with reference toFIGS.5-10 and15-16, an application (e.g.,160) may be configured to access and/or operate one or more cameras, process one or more images, and/or perform one or more image processing operations, such as object detection, object recognition, object tracking (using a marker or marker-less), or a combination thereof, as illustrative, non-limiting examples. Additionally, as described herein, the camera video stream may be presented in real-time or near real-time such that auser viewing display172 does not perceive a noticeable delay between a change in position and/or orientation of a camera and presentation of the camera video stream (produced based on a field of view of the camera). In some implementations, a delay in presentation of the camera video stream may be noticeable and may still be considered near real-time. To illustrate, such delays may be based on a processing capability and/or image rendering capability of a device (e.g.,110,150) that includes the camera. It is noted that one or more frames of and/or one or more operations described with reference to one ofFIGS.5-10 and15-16 may be combined with one or more frames of and/or one or more operations of another ofFIGS.5-10 and15-16. For example, one or more frames ofFIG.5 may be combined with one or more frame ofFIG.6. As another example, one or more frames ofFIGS.5-6 may be combined with the frame ofFIG.8. As another example, one or more frames ofFIGS.9A-9D may be combined with the frame ofFIG.7. As another example, one or more frames ofFIG.5-6 or9A-9D may be combined with one or more frames ofFIG.10.
• Referring toFIG.5, an example of a series offrames500 presented viadisplay172 is shown. Afirst frame510 of theseries500 displays atarget512. In some implementations,target512 may be presented based on accessingapplication160, detectingidentifier187, communicatingidentifier data197 toserver130, receivingvideo content193, or a combination thereof.Target512 may be configured to aid a user in aligning a marker (e.g.,188) within a field of view of a camera. In some implementations,target512 may include a bracket, line, and/or border to frame (e.g., outline) the marker, a transparent representation of the marker, or a combination thereof, as illustrative, non-limiting examples.
• Asecond frame520 ofseries500 includestarget512 positioned overmarker188 ofpackage186. For example,package186 may be included in a camera video stream (e.g.,170) presented viadisplay172.Application160 may perform object recognition/detection onsecond frame520 to identifymarker188. Based on detectingmarker188, the application may determine a position ofmarker188, establish a coordinate system based onmarker188, identify one or more edges of box (e.g.,186), perform object tracking onmarker188 and/orpackage186, or a combination thereof. Additionally, or alternatively, in response to detection ofmarker188,application160 may removetarget512 fromdisplay172 and/or may initiate presentation of the multimedia message includingAR content162 andvideo content193.
• Athird frame530 ofseries500 includesAR content162, such as a 3D object (e.g., a 3D animated character), withinthird frame530. The character is displayed at a first position within the camera video stream (e.g.,170) that includespackage186. To illustrate, betweensecond frame520 andthird frame530, the character may move to the first position onpackage186. As illustrative, non-limiting examples, the character may have appeared out of thin air, dropped down ontopackage186, parachuted ontopackage186, or climbed ontopackage186. Afourth frame540 ofseries500 shows the character (e.g.,162) at a second position within the camera video stream. For example, betweenthird frame530 andfourth frame540, the character may have walked from the first position towards the second position and tripped overmarker188. In other implementations, the character may have tripped over another objected included in the camera video stream, such as a ribbon (not shown) that is coupled to or incorporated intopackage186.
• Afifth frame550 ofseries500 includes the character at a third position within the camera video stream andadditional AR content162, such as a monitor (e.g., a television), at a fourth position within the camera video stream. Infifth frame550, the character includes a remote control in one hand. To illustrate, betweenfourth frame540 andfifth frame550, the character may have produced the remote control and operated the remote control to cause the monitor to be included (e.g., drop down onto or raise up from package186) in the camera video stream. The monitor may have ascreen552 that is associated with or corresponds tovideo content193 to be displayed. In some implementations, a boundary/perimeter ofscreen552 may include or correspond to theplacement data163.
• Asixth frame560 ofseries500 includes the AR content162 (e.g., a character and monitor) included within the camera video stream.Sixth frame560 also includesvideo content193 within camera video stream and positioned based onAR content162. For example,video content193 is positioned within the monitor such thatvideo content193 is presented viascreen552 of the monitor. In some implementations, a light source may564 may be positioned at the screen and radiate light outward from the screen. To further illustrate, betweenfifth frame550 andsixth frame560, the character may have operated the remote control to causevideo content193 to be presented viascreen552 of the monitor.
• Thus,FIG.5 illustrates aspects of initiation and presentation of the multimedia message viadisplay172. The multimedia message may present theAR content162 in the camera video stream. Additionally, the multimedia message may present thevideo content193 in relation to the AR content.
• Referring toFIG.6, an example of a series offrames600 presented viadisplay172 is shown. Afirst frame610 of theseries600 is associated with a camera video stream (e.g.,170) and includescard410 havingidentifier187. Based onfirst frame610,application160 may detectidentifier187 and sendidentifier data197 toserver130. In other implementations,identifier187 may be coupled topackage186 andfirst frame610 may includeidentifier187 coupled topackage186. Responsive to sendingidentifier data197 toserver130,mobile device150 may receiveAR content162,video content193, and/or other content (e.g., text data and/or image data) fromserver130.
• Asecond frame620 of theseries600 includespackage186 includingmarker188. Prior tosecond frame620,application160 may have detected and identifiedmarker188. For example,first frame510 andsecond frame520 ofseries500 ofFIG.5 may have occurred betweenfirst frame610 andsecond frame610 ofseries600.
• Athird frame630 ofseries600 includesAR content162, such as a ladder, withinthird frame630. To illustrate, betweensecond frame620 andthird frame630, the ladder may have propped against a side ofpackage186. Afourth frame640 ofseries600 showsAR content162 including the ladder and the character. To illustrate, betweenthird frame630 andfourth frame640, the character may have climbed up a portion of the ladder towards a top surface ofpackage186. Afifth frame650 ofseries600 includes the character at a first position on the top surface ofpackage186. To illustrate, betweenfourth frame640 andfifth frame650, the character may have climb off of the ladder onto the top surface ofpackage186 and moved to the first position. Asixth frame660 ofseries600 includes theAR content162, such as the ladder, the character, and the monitor.Sixth frame660 also includesvideo content193 positioned based onAR content162. For example,video content193 is positioned within the monitor such thatvideo content193 is presented via a screen of the monitor. To further illustrate, betweenfifth frame650 andsixth frame660, the monitor may have been included in the camera video stream and character may have moved to a second position as shown insixth frame660.
• Thus,FIG.6 illustrates aspects of initiation and presentation of the multimedia message viadisplay172. For example,FIG.6 illustrates detection ofidentifier187 and presentation ofAR content162 in the camera video stream. Additionally,FIG.6 illustrates presentation of thevideo content193 based on (e.g., within) theAR content162.
• Referring toFIG.7, an example of aframe700 included in a series of frames (associated with a multimedia message) presented viadisplay172 is shown. As shown,frame700 includespackage186AR content162, such as a character and a monitor (e.g., a television).Video content193 is positioned within the monitor such thatvideo content193 is presented via a screen of the monitor. TheAR content162 also includes a sign702 (e.g., a card). Sign702 includes one or more portions, such as afirst portion710 including text content and asecond portion712 including an image, such as a logo, picture, graphic, etc. The content included infirst portion710,second portion712, or both, may include or correspond tomessage field256 and ormessage ID322. Sign702 may be included in the camera video content prior to, during (e.g., a portion or an entirety), and/or after presentation ofvideo content193. Thus,FIG.6 illustrates aspects of presentation of the multimedia message viadisplay172.
• Referring toFIG.8, an example of aframe800 included in a series of frames (associated with a multimedia message) presented viadisplay172 is shown. As shown,frame800 includespackage186 andAR content162, such as a character and a sign802 (e.g., a card). Sign802 includes one or more portions, such as afirst portion810, asecond portion812, and athird portion814.First portion810 includes text content,second portion812 includes an image, such as a logo, picture, graphic, etc., andthird portion814 includesvideo content193. The content included infirst portion810,second portion812, or both, may include or correspond tomessage field256 and ormessage ID322. Thus,FIG.7 illustrates aspects of presentation of the multimedia message viadisplay172.
• Referring toFIGS.9A-9D, examples ofAR content162 in whichvideo content193 is presented are shown. To illustrate, each of the examples ofFIGS.9A-9D enable at least aportion video content193 to be viewed over a range of camera positions and/or camera orientations. For example, each ofFIGS.9A-9C include a first frame associated with a first camera pose (e.g., a first position and a first orientation ofcamera168 with respect to package186 (e.g., marker) and a second frame associated with a second camera pose (e.g., a second position and a second orientation ofcamera168 with respect to package186 (e.g., marker188). In some implementations, the first camera pose is associate with a front view ofpackage186 and the second camera pose is associated with a rear view ofpackage186. In each of the first frame and the second frame, a portion and/or version ofvideo content193 is visible to a user such that movement/positioning of the camera by the user is not limited in order to view thevideo content193 of the multimedia message.
• Referring toFIG.9A, an example of a series of frames900 (associated with camera video stream170) presented viadisplay172 is shown. Afirst frame910 of theseries900 includes a frame of camera video stream (e.g.,170) captured by camera (e.g.,168) at a first position and first orientation with respect to package186 (e.g., marker188). For example,first frame910 may correspond to a first camera pose associated withcamera168.First frame910 includesAR content162, such as the character, a projector, and a projection screen. As shown, a front view ofvideo content193 is presented on the projection screen that is flat. In some implementations,first frame910 may include a virtuallight source912 configured to provide light (e.g., a blinking light) within the multimedia content. The video content may be positioned and/or sized in relation to the AR content162 (e.g., the projection screen) based on the first camera pose, such as the first position and orientation ofcamera168.
• Asecond frame920 of theseries900 includes another frame of camera video stream captured bycamera168 at a second position and a second orientation with respect to package186 (e.g., marker188). For example,second frame920 may correspond to a second camera pose associated withcamera168.Second frame920 includesAR content162, such as the character, the projector, and the projection screen. As shown,video content193 is presented on the projection screen. As compared tofirst frame910,video content193 displayed insecond frame920 is a reverse image (e.g., a rear view) ofvideo content193 displayed infirst frame910.
• Referring toFIG.9B, an example of a series of frames900 (associated with camera video stream170) presented viadisplay172 is shown. Afirst frame940 of theseries930 is associated with the first camera pose and asecond frame950 of theseries930 is associated with the second camera pose. Each of the first andsecond frames940,950 includesAR content162, such as the character, the projector, and the projection screen. As shown, the projection screen has a curved surface. In other implementations, the projection screen may have different size, shape, and/or geometry. For example, the projection screen may be disc shaped (e.g., semi-spherical).
• Referring toFIG.9C, an example of a series of frames960 (associated with camera video stream170) presented viadisplay172 is shown. Afirst frame970 of theseries960 is associated with the first camera pose and asecond frame980 of theseries960 is associated with the second camera pose. Each of the first andsecond frames970,980 includesAR content162, such as the character and a three-dimensional (3D) structure of multiple screens. For example, as shown infirst frame970, the multi-screen structure includes a front screen and a side screen. In some implementations, atop surface972 of the multi-screen structure may include text content or an image (e.g., a logo). As shown insecond frame980, the multi-screen structure includes the side screen and a rear screen. Each of the screens of the multi-screen structure may includevideo content193. To illustrate,video content193 presented via each screen of the multi-screen structure may be synchronized in time. In some implementations, each of the multiple screens may include a corresponding virtuallight source973 configured to provide light within the multimedia content.
• Referring toFIG.9D, an example of at least one frame of a series offrames995 of multimedia content (associated with camera video stream170) presented viadisplay172 is shown. Aframe996 of theseries995 is associated with a first camera pose and includesAR content162, such as the character, the film projector, and the screen. As shown,video content193 is projected from the film projector onto one or more objects, such as the character and the screen. In some implementations, thevideo content193 includes a modified version of thevideo193 generated based on at least one mask (e.g.,163). When the character is positioned between the film projector and the screen, a first portion ofvideo content193 is projected onto the screen and asecond portion997 ofvideo content193 is projected onto the character. Additionally,light source912 positioned at the film projector creates a light such that the character casts ashadow998 on the screen in an area corresponding to thesecond portion997 of the video content. In some implementations,light source192 is configured to blink, thereby creating a realistic effect of a film projector. Additionally, or alternatively, one or more audio sources may be included such that the multimedia content includes first audio (e.g., character noise/voice, film projector noise, etc.) associated with theAR content162 and second audio associated with thevideo content193. In some implementations,frame996 may be generated based on one or more rules that correspond to execution of an instruction by application160 (e.g., processor154). Thus,FIG.9D depicts implementation of theAR content162 withvideo content193 displayed on the screen by film projector in a realistic manner, such that a portion of another object (e.g., the character) may obstruct at least a portion ofvideo content193 to be displayed on the screen. Accordingly, as described herein,application160 provides an integrated approach embodied in computer software for the rapid, efficient manipulation and presentation ofvideo content193 in manner that creates a realistic effect of thevideo content193 being displayed via the film projector (e.g.,162).
• Referring toFIG.10, an example of a series offrames1000 presented viadisplay172 is shown. The series offrames1000 may be generated by one ormore processors154. In some such implementations,processor154 includes a first processor to process first image data from a first camera (e.g., a forward facing camera) and a second processor to process second image data from a second camera (e.g., a rear facing camera). The first processor, the second processor, or another processor may be configured to combine and/or initiate presentation of the processed first image data, the processed second image data, or a combination thereof.
• Afirst frame1010 of theseries1000 displays target512 in a camera video stream corresponding to a first camera (e.g.,168).Target512 may be configured to enable a user to align a marker (e.g.,188) within a field of view of the first camera.First frame1010 also includes auser portion1012 associated with a second camera video stream of a second camera. To illustrate,mobile device150 may include multiple camera, such as a front facing camera (e.g., first camera) and a rear facing camera (e.g., second camera). As shown,user portion1012 includes a target (e.g., dashed line).
• Asecond frame1020 ofseries1000 includestarget512 positioned overmarker188 ofpackage186. For example,package186 may be included in a camera video stream (e.g.,170) presented viadisplay172. An application (e.g.,160) may perform objection recognition/detection onsecond frame520 to identifymarker188. Based on detectingmarker188, the application may determine a position ofmarker188, establish a coordinate system based onmarker188, identify one or more edges of box (e.g.,186), perform object tracking onmarker188 and/orpackage186, or a combination thereof. Additionally, as shown, a face of the user is includeduser portion1012. In some implementations,application160 may detect that an object (e.g., a face) is present in second camera video stream. Based on identification ofmarker188 and/or detection of bothmarker188 and the object (e.g., a face),application160 may initiate presentation of AR content (e.g.,162) and/or video content (e.g.,193) within first camera video stream (e.g.,170). In some implementations, based on identification ofmarker188 and/or detection of bothmarker188 and the object (e.g., a face),application160 may initiate a first recording corresponding to presentation of AR content (e.g.,162) and/or video content (e.g.,193) within first camera video stream (e.g.,170). Additionally, or alternatively, based on identification ofmarker188 and/or detection of bothmarker188 and the object (e.g., a face),application160 may initiate a second recording corresponding to second camera video stream.
• Athird frame1030 ofseries1000 includesAR content162, such as a 3D character at a first position, withinthird frame1030. To illustrate, betweensecond frame520 andthird frame530,target512 and the target ofuser portion1012 have been removed, and the character may be presented onpackage186. As compared tosecond frame1020,user portion1012 has a reduced size to make more of the first camera video stream (and AR content) visible viadisplay172.
• Afourth frame1040 ofseries1000 showsAR content162 that includes character (at a second position) and a monitor (e.g., a television) within the first camera video stream. Infourth frame1040, the character includes a remote control in one hand andvideo content193 is presented and positioned based onAR content162. For example,video content193 is positioned within the monitor such thatvideo content193 is presented via a screen of the monitor.
• Afifth frame1050 ofseries1000 shows the character at a third position. To illustrate, betweenfourth frame1040 andfifth frame1050, the character moved from the second position (e.g., a lower left portion of display172) to the third position (e.g., a lower right portion of display172). Based on the movement of the character betweenfourth frame1040 andfifth frame1050,application160 may have adjusted a characteristic ofuser portion1012. For example, the characteristic ofuser portion1012 may include a size, a shape, a position, a location, a transparency, another characteristic, or a combination thereof. As shown infifth frame1050,user portion1012 is positioned in the lower right ofdisplay172 as compared to being displayed in the lower right ofdisplay172 infourth frame1040. In some implementations,application160 may adjust the characteristic ofuser portion1012 in response to a determination that at least a portion of or an entirety of the character is within a threshold a threshold distance ofuser portion1012.
• Asixth frame1060 ofseries1000 includes the AR content162 (e.g., a character and a sign1064).Sixth frame1060 may be presented after completion of thevideo content193. In some implementations,sixth frame1060 may be presented afterapplication160 stopped storing the first recording and/or the second recording. Thesign1064 may include aportion1066 that includes a message for the user. For example, the message may indicate an option for the user to provide a reply message to a sender ofpackage186. To illustrate, the replay message may include the first recording, the second recording, a text message (e.g., a static message or a dynamic/scrolling message), a video message recorded by the user (e.g., a pre-recorded or existing video), or a video recorded after presentation of the multimedia content including theAR content162 and the video content193), an audio clip, or a combination thereof. In a particular implementation,application160 may control the second camera anddisplay172 to enable the user to record a reply video message for the sender.
• Thus,FIG.10 illustrates aspects of presentation of the multimedia message viadisplay172. For example, as described with reference toFIG.10, first and/or second recordings may be generated in association presentation of the multimedia content. The first and/or second recordings may be available to the user (e.g., recipient) for replay or transmission/posting for another user (i.e., sender). In some implementations, an option to record an additional message and/or video content may be presented to the user (e.g., recipient).
• Referring toFIG.15, an example of a series offrames1500 presented viadisplay172 is shown. Afirst frame1510 of theseries1500 includespackage186 includingmarker188. Prior tofirst frame1510,application160 may have detected and identifiedmarker188.
• Asecond frame1520 ofseries1500 includesAR content162, such as a saw, withinsecond frame1520. To illustrate, betweenfirst frame1510 andsecond frame1500, the saw may extended through a surface ofpackage186. At least a portion of the saw may be occluded based on a 2D virtual plane (not shown). Although the saw is described as being AR content, in other implementations, the saw may be presented as animated content.
• Athird frame1530 ofseries1500 includes anopening1532 in the surface ofpackage186. For example, opening1532 may have been created by saw betweensecond frame1520 andthird frame1530.
• Afourth frame1540 ofseries1500 includesAR content162, such as a 3D character and a ladder that extend through surface of package186 (e.g., through opening1532). For example, betweenthird frame1530 andfourth frame1540, a portion of the ladder may have been positioned throughopening1532 and the character may have climbed towards a top of the ladder.
• Afifth frame1550 ofseries1500 includes the character on the surface ofpackage186 at a third position within the camera video stream andadditional AR content162, such as a monitor (e.g., a television), at a fourth position within the camera video stream. For example, betweenthird frame530 andfourth frame540, the character may have walked from the first position towards the second position and tripped overmarker188.
• Asixth frame1560 ofseries1500 includes theAR content162, which includes the character, the ladder, and a monitor.Sixth frame1560 also includesvideo content193 within camera video stream and positioned based onAR content162. For example,video content193 is positioned within the monitor such thatvideo content193 is presented the monitor.
• Thus,FIG.15 illustrates aspects of initiation and presentation of the multimedia message viadisplay172. The multimedia message may present theAR content162 in the camera video stream. For example, one or more features of theAR content162 may be presented using one or more virtual objects (not shown), as described with reference toFIGS.17-18. To illustrate, the one or more virtual objects may enable occlusion of theAR content162 such that the AR content appears to extend out ofpackage186 and is visible from a variety of camera poses.
• Referring toFIG.11, an example of a series of frames1100 presented via adisplay1112 is shown. The series offrames1000 may be generated by one ormore processors154. In some such implementations,processor154 includes a first processor to process first image data from a first camera (e.g., a forward facing camera) and a second processor to process second image data from a second camera (e.g., a rear facing camera). The first processor, the second processor, or another processor may be configured to combine and/or initiate presentation of the processed first image data, the processed second image data, or a combination thereof.
• Display1112 may include or correspond to a display of a device, such aselectronic device110 ormobile device150. The series of frames1100 may include or correspond to playback of recorded content, such as content recorded at the same device or a different device. For example, the recorded content ma include or correspond tomobile device content199.
• Afirst frame1110 of the series1100 includes afirst portion1114 and asecond portion1116.First portion1114 may include or correspond to a first recording, such as a recording of multimedia content presented atmobile device150.Second portion1116 may include or correspond to a second recording, such as recording of a user ofmobile device150 during presentation of the multimedia content atmobile device150. For example, the second recording may include or correspond to content displayed viauser portion1012. As shown,first frame1110 shows first scenes of each of the first recording (presented in first portion) and second recording (presented in second portion). In some implementations, presentation of the first and second recordings viadisplay1112 may be time synchronized.
• Asecond frame1120 of series1100 shows second scenes of each of the first recording (presented in first portion) and second recording (presented in second portion). For example, as shown,second frame1120 shows the character (in first portion1114) after the character tripped, and shows a reaction of the user (in second portion1116) responsive to observing that the character tripping. Athird frame1130 of series1100 shows third scenes of each of the first recording (presented in first portion) and second recording (presented in second portion). For example, as shown,third frame1130 shows the character and the monitor including video content (in first portion1114), and shows the user (in second portion1116). Afourth frame1140 of series1100 shows fourth scenes of each of the first recording (presented in first portion) and second recording (presented in second portion). In some implementations, the fourth scenes may be associated with an end of the first and second recordings.
• Afifth frame1150 of series1100 shows a recorded video message from the user (e.g., the receiver) to the sender. For example, the message may have been recorded after the user viewed the multimedia content. AlthoughFIG.11 has been described as presenting three different recordings, in other implementations, fewer than three or more than three recordings may be presented.
• Thus,FIG.11 illustrates describes playback of recorded content associated with the multimedia message. To illustrate, a first recording presented viafirst portion1114 includes the multimedia message as presented to the recipient and second recording presented viasecond portion1116 include the recipient's demeanor/reaction during presentation of the multimedia message. The third recoding presented after the first and second recordings includes a message from the recipient to the sender.
• Referring toFIG.16, an example of a series offrames1600 presented viadisplay172 is shown. As shown,display172 includes afirst portion1612 and asecond portion1614. Thefirst portion162 include multimedia content and thesecond portion1614 includes auser interface1616.User interface1616 may include a selectable button to activate and/or deactivate recording of the multimedia content presented via thefirst portion1612.First portion1612 may be offset from a bottom edge ofdisplay172 andsecond portion1614 may be positioned between the bottom edge ofdisplay172 and a bottom edge offirst portion1612.
• Afirst frame1610 of theseries1600 displays target512 viafirst portion1612. In some implementations,target512 may be presented based on accessingapplication160, detectingidentifier187, communicatingidentifier data197 toserver130, receivingvideo content193, or a combination thereof.Second portion1614 includesuser interface1616 to enable a user to initiate recording of multimedia content presented viafirst portion1612.
• Asecond frame1620 ofseries1600 includestarget512 positioned overmarker188 ofpackage186. Betweenfirst frame1610 andsecond frame1620,user interface1616 may have been selected to initiate recording. Accordingly, as shown insecond frame1620,user interface1616 indicates that recording is pending. Whenuser interface1616 indicates that recording is pending, selection ofuser interface1616 may cancel the pending recording.Application160 may perform object recognition/detection onsecond frame1620 to identifymarker188. Based on detectingmarker188,application160 may removetarget512 fromdisplay172 and/or may initiate presentation of the multimedia message (e.g., the multimedia content) includingAR content162 andvideo content193. In addition to initiating presentation of the multimedia message,application160 may initiate recording to start upon presentation of the multimedia message.
• Athird frame1630 ofseries1600 includesAR content162, such as a 3D animated character, withinthird frame1630. Betweensecond frame1620 andthird frame1630,application160 initiates multimedia message and recoding of the multimedia message. Accordingly, as shown insecond frame1630,user interface1616 indicates that recording. In some implementations,user interface1616 may indicate a duration of the recording. When recording,application160 may record a portion or all ofdisplay172. For example,application160 may recordfirst portion1612 but notsecond portion1614. Whenuser interface1616 indicates recording, selection ofuser interface1616 may stop the recording.
• Thus,FIG.16 illustrates aspects of initiation and presentation of the multimedia message viadisplay172.FIG.16 further illustratesuser interface1616 to enable recoding of the multimedia message.
• Referring toFIGS.17 and18, examples of techniques for occlusion of AR content are shown. For example,FIG.17 illustrates a first technique for occlusion of AR content andFIG.18 illustrates a second technique for occlusion of AR content. The first technique (ofFIG.17) or the second technique (ofFIG.18) may be implemented may implemented by a device configured to generate AR content, such aselectronic device110,server130, mobile device150 (e.g.,processor154 and/or application160), an AR engine, or another device. For example, the first technique (ofFIG.17) or the second technique (ofFIG.18) may be used to generate and/or provide multimedia content, such as multimedia content described herein with reference toFIGS.1,5-8,9A-9D,10-11, and15-16.
• Referring toFIG.17, the first technique is describes as series of stages. It is noted that the series of stages ofFIG.17 are presented for illustration purposes and depictions of one or more stages may not be presented via a display (e.g.,172). Afirst stage1700 illustratespackage186 of a real world environment. Asecond stage1710 includes virtual objects include in a virtual environment. For example, the virtual objects may be generated by mobile device150 (e.g.,processor154 and/or application160). The virtual objects may be positioned within a coordinate system established based on package186 (e.g., marker188). As shown, the virtual objects include a2D plane1712, a hole1714 (or opening) defined in the2D plane1712, and acontainer1716. As shown,container1716 includes a cylinder, however, in other implementations,container1716 may include a different shape.Container1716 may include an opening that is aligned withhole1714. In some implementations, an opening ofcontainer1716 may be larger thanhole1714. The virtual objects1712-1716 may be configured to be used for occlusion, collision detection, or both.
• Referring to athird stage1730, AR content1732 (e.g., a ladder) is shown within the virtual environment that includes virtual objects1712-1716. During generation of AR content (e.g.,162) within a live video stream, a camera pose of an image capture device (e.g.,168) may be determined and at least a portion of theAR content1732 to be included in the live video stream may be determined based on the camera pose and the virtual objects1712-1716. To illustrate, based on a determined camera pose, a determination may be made as to one or more portions of theAR content1732 to be occluded and one or more portions of theAR content1732 to be included in the live video stream.
• Referring to afourth stage1740, the virtual objects1712-1716 (of the virtual environment) are depicted in relation to the package186 (of the real world environment) to illustrate use of the virtual objects1712-1716 to generate theAR content1732 in the real world environment. Referring to afifth stage1750, thepackage186 is shown along with theAR content1732 that extends from an opening1752 (additional AR content). The portion of theAR content1732 was determined based on the virtual objects1712-1716 and the camera pose, and include in the live video stream of the real world environment.
• Thus, the stages shown with reference toFIG.17 illustrate use of one or more virtual objects to determine a portion of theAR content1732 to include in the live video stream. The virtual objects may enable one or more features of theAR content162 to be occluded using one or more virtual objects1712-1716. The virtual objects1712-1716 may enable occlusion for one or more packages (e.g.,186) without having a specific 3D model for each of the one or more packages. Accordingly, the first technique enables uses of asingle marker188 for multiple packages, such as multiple packages having different physical dimensions. Additionally, or alternatively, the first technique enables reduced storage of virtual objects to enable occlusion as compared to techniques that utilize 3D models of real world objects. The first technique also enables reduced processing complexity to perform occlusion because the virtual objects used for occlusion are often less complex than a 3D model of the real world objects and, therefore, are easier to track and manipulate for different camera poses.
• Referring toFIG.18, the second technique is describes as series of stages. It is noted that the series of stages ofFIG.18 are presented for illustration purposes and depictions of one or more stages may not be presented via a display (e.g.,172). As shown, each of the stages shown inFIG.18 includes one or more virtual objects1804-1810 and arepresentative camera pose1816. The camera pose1816 may include or correspond to a position ofmobile device150 with respect to a real world object, such aspackage186. The one or more virtual objects1804-1810 include a first 2D plane1804 (associated with a surface of package186), anopening1806 in thefirst 2D plane1804, anaxis1808 that extends through a center ofopening1806, and asecond 2D plane1810. Second 2D plane has atop edge1812 that is co-planer with first 2D plane. At least one virtual object (e.g.,1810) is dynamically positioned with respect to one or more othervirtual objects1804,1806,1808 based oncamera pose1816. To illustrate, second 2D plane is configured to rotate (e.g., be positioned) aboutaxis1808 based on a position of camera pose1816, as described further herein. Although the at least one virtual object (e.g.,1810) is depicted as a 2D plane, in other implementations, the at least one virtual object (e.g.,1810) may be any shape (e.g., curved) and/or may be a three dimensional. Accordingly, AR content (e.g.,162,1732) that is positioned to extend out of opening1806 (and/or package186) may be occluded based on one or more of thevirtual objects1804,1806,1810.
• Referring to afirst stage1800, camera pose1816 is determined at a first position (with respect topackage186 and/or marker188). As shown, based on camera pose1816 at the first position,second 2D plane1810 is positioned aboutaxis1808 such thatsecond 2D plane1810 is positioned betweenaxis1808 and camera pose1816.
• Referring to asecond stage1830, camera pose1816 is determined at a second position (with respect topackage186 and/or marker188). As shown, based on camera pose1816 at the second position,second 2D plane1810 is positioned aboutaxis1808 such thatsecond 2D plane1810 is positioned betweenaxis1808 and camera pose1816.
• Referring to athird stage1850, camera pose1816 is determined at a third position (with respect topackage186 and/or marker188). As shown, based on camera pose1816 at the third position,second 2D plane1810 is positioned aboutaxis1808 such thatsecond 2D plane1810 is positioned betweenaxis1808 and camera pose1816.
• Thus, the stages shown with reference toFIG.18 illustrate use of one or more virtual objects to determine a portion of the AR content to include in the live video stream. The virtual objects may enable one or more features of theAR content162 to be occluded using one or more virtual objects1804-1810. The virtual objects1804-1810 may enable occlusion for one or more packages (e.g.,186) without having a specific 3D model for each of the one or more packages. Accordingly, the second technique enables uses of asingle marker188 for multiple packages, such as multiple packages having different physical dimensions. Additionally, or alternatively, the second technique enables reduced storage of virtual objects to enable occlusion as compared to techniques that utilize 3D models of real world objects. The second technique also enables reduced processing complexity to perform occlusion because the virtual objects used for occlusion are often less complex than a 3D model of the real world objects and, therefore, are easier to track and manipulate for different camera poses.
• Referring toFIG.12, amethod1200 of operating an electronic device is shown. For example,method1200 may be performed byelectronic device110,mobile device150, or electronic terminal182.
• Method1200 may include initiating, at the electronic device, an order request via a server, at1502. For example, the order request may include or correspond toorder form210. The server may include or correspond toserver130.
• Method1200 includes generating order information by the electronic device, at1204. For example, the order information may include or correspond to orderinformation192, input(s) provided to one or more fields242-258, or a combination thereof. To illustrate, generating (e.g.,1204) the order information may include selecting an item, at1206, providing sender information and recipient information, at1208, and requesting a multimedia message, at1210. Requesting (e.g.,1210) the multimedia content may include selecting augmented reality content, at1212, providing video content, at1214, providing a text message, at1216, providing an image, at1218, or a combination thereof. For example, the augmented reality content and the video content may include or correspond toAR content162 andvideo content193, respectively.
• Method1200 may include receiving, at the electronic device, a multimedia message preview from the server, at1220. The multimedia message preview may include or correspond to previewcontent139.Method1200 includes submitting, by the electronic device, the order request to the server, at1222. The order request may include or correspond to orderinformation192 and/ororder form210.
• Method1200 may include receiving, at the electronic device, a delivery confirmation, at1224. The delivery confirmation, such asdelivery confirmation196, may correspond to delivery of a package that includes at least one item identified in the order request. For example, the package may include or correspond to package186 delivered bydistribution entity180.
• Method1200 may include receiving, at the electronic device, mobile device content associated with the multimedia message, at1226. For example, the mobile device content may include or correspond tomobile device content199 and/or series of frames1100.Method1200 may include presenting the mobile device content via a display device of the electronic device, at1228. Thus,method1200 may enable operation of an electronic device (e.g.,110,150) to order delivery of an item to a recipient and to be notified of delivery of multimedia content associated with the delivery.
• Referring toFIG.13, amethod1300 of operating a server is shown. For example,method1300 may be performed byserver130.Method1300 may include receiving, by the server from an electronic device, an order request for an item, at1302. For example, the order request may include or correspond to theorder information192 ororder form210. The electronic device may include or correspond toelectronic device110,mobile device150, or electronic terminal182. The order request may be associated with video content, such asvideo content193.
• Method1300 may include initiating storage of the video content by the server, at1304. To illustrate, the server may initiate storage of the video content at a memory, such asmemory136, that is accessible to the server. In some implementations, the server may receive the video content from a device, such the electronic device that that initiated the order request. In other implementations, the server may receive the video content from a manufacturer, provider, supplier, and/or distributor of the item, a delivery service or shipper of a package that includes the item, or another device or entity.
• Method1300 may include generating an order entry by the server, at1306. For example, the order entry may include or correspond toentry142. Generating (e.g.,1306) the order entry may include generating an order number, at1308, generating an identifier corresponding to the order number, at1310, or both. The order number may include or correspond toorder number310. The identifier may include or correspond toidentifier187,marker188, or a combination thereof.
• Method1300 may include providing, by the server, an order request to a distribution entity, at1312. For example, the order request may include or correspond to orderrequest195, and the distribution entity may include or correspond todistribution entity180.Method1300 may include receiving, by the server, confirmation of delivery of a package correspond to the order request, at1314. For example, the confirmation may include or correspond todelivery confirmation196. In some implementations, the server may receive the confirmation fromdistribution entity180 and/or may send the confirmation to a device, such as a device that initiated the order request.
• Method1300 may include receiving, by the server, identifier data from a mobile device, at1316. For example, the identifier data may include or correspond toidentifier data197, and the mobile device may include or correspond tomobile device150. The identifier data may include or correspond toidentifier187.Method1300 includes identifying, by the server, the order entry based on the identifier data, at1318. To illustrate, the server may identify and/oraccess entry142 based on identifier data.Method1300 further includes initiating, by the server, transmission of the video content to the mobile device, at1320.
• Method1300 may also include receiving, at the server from the mobile device, a notification of presentation of a multimedia message, at1322. For example, the notification may include or correspond tonotification198. The multimedia message may be associated with the order request and may include the video content and augmented reality content, such as 3D augmented reality content. To illustrate, multimedia message may include or correspond tomobile device content199. In some implementations, the notification includesmobile device content199. In such implementations, the server may provide the notification and/or themobile device content199 to a device, such as a device that initiated the order request. Thus,method1300 describes operation of server (e.g.,130) to enable delivery of an ordered item to a recipient, delivery of multimedia content to the recipient, and notification of delivery of the multimedia content.
• Referring toFIG.14, amethod1400 of operating a mobile device is shown. For example,method1400 may be performed byelectronic device110,mobile device150, or electronic terminal182.
• Method1400 includes receiving, at a mobile device from a server, video content corresponding to an identifier, at1402. For example, the server may include or correspond toserver130, and the video content may include thevideo content193. The identifier may be associated with a package. To illustrate, the identifier may include or correspond toidentifier187 associated withpackage186.
• Method1400 include receiving, at the mobile device, a plurality of images including at least a portion of the package, at1404. The plurality of images may correspond to a camera video stream. For example, the plurality of images may include or correspond to the one ormore images170.
• Method1400 includes generating, at the mobile device, multimedia content associated with the package,1406. Generating (e.g.,1406) may include displaying the camera video stream by the mobile device, at1408. Generating (e.g.,1406) may also include displaying, by the mobile device, augmented reality content within at least a portion of the displayed camera video stream, at1410, and displaying, by the mobile device, the video content in relation to the augmented reality content, at1412. The augmented reality may include or correspond toAR content162.
• In some implementations, the multimedia content associated with the package may be generated at a different device, such as at server103. In such implementations, the sever may transmit (e.g., stream) the multimedia content to the mobile device for presentation via a display.
• Thus,method1400 describes operation of a device (e.g.,110,150) to enable delivery of a multimedia message that corresponds to delivery of a package (e.g., an item). The multimedia message may advantageously display AR content and video content in a live video stream generated by a camera of the device.
• Referring toFIG.19, amethod1900 of operating a mobile device is shown. For example,method1400 may be performed byelectronic device110,mobile device150, or electronic terminal182.
• Method1900 includes receiving, at a mobile device from a server, video content associated with a plurality of frames, at1902. For example, the server may include or correspond toserver130, and the video content may include thevideo content193. The video content may be associated with a package (e.g.,186). In some implementations,method1900 includes buffering the one or more frames of the video content.
• Method1900 include applying, at the mobile device, a mask to one or more frames to generate modified video content, at1904. The mask may include or correspond to mask136. To illustrate, the mask may be configured to blur one or more edges of the one or more frames to create a falloff effect. Althoughmethod1900 is described as generating modified video content, in some implementations, the video content may not be modified and me presented as received by the mobile device.
• Method1900 includes generating, at the mobile device, multimedia content,1906. The multimedia content may be associated with the package (e.g.,186). Generating (e.g.,1906) may include displaying a camera video stream by the mobile device, at1908. For example, the camera video stream may be generated by an image capture device of the mobile device and may include an object, such aspackage186. Generating (e.g.,1906) may also include displaying, by the mobile device, augmented reality content within at least a portion of the displayed camera video stream, at1910. The augmented reality content may include one or more virtual objects, such as a first virtual object and a second virtual object.
• Generating (e.g.,1906) may also include displaying, by the mobile device, the modified video content based on the first virtual object onto the second virtual object, at1912, and providing a virtual light source during display of the modified video content, at1914. The virtual light source may be associated with the first virtual object and may be positioned to correspond with presentation of the version of the video content. In some implementations, the virtual light source may be configured to blink during presentation of the modified video content.
• In some implementations, the modified video content and/or the multimedia content associated with the package may be generated at a different device, such as at server103. In such implementations, the sever may transmit (e.g., stream) the modified video content and/or the multimedia content to the mobile device for presentation via a display.
• In some implementations, the object includes a real world object and/or the first virtual object comprises a film projector or a monitor. Additionally, or alternatively, themethod1900 may providing a user interface concurrently with display of the camera stream. The user interface configured to enable recording of the camera stream including the version of the video content. For example, the user interface may include or correspond touser interface1616.
• In some implementations,method1900 may include displaying the second virtual object in the camera stream. In a particular implementation, when the first virtual object includes the film projector, the virtual light source may be positioned at the first virtual object, the second virtual object may include a projector screen, and a version of the video content may be projected from the film projector toward the second virtual object. Alternatively, when the first virtual object includes the monitor, the virtual light source may be positioned within the first virtual object, and a version of the video content may be project from within the first virtual object onto a screen of the monitor.
• In some implementations,method1900 includes detecting a marker or the object in the camera stream and establishing a coordinate system based on the detected mark or the detected object. The coordinate system may be associated with a real world environment or a virtual environment.Method1900 may also include: positioning, based on the coordinate system, a two dimensional (2D) plane in a virtual environment; determining an opening in the 2D plane; determining, based at least on the 2D plane, the opening, or both, a first portion of a third virtual object and a second portion of the third virtual object; and presenting the first portion of the third virtual object in the camera video stream based on the coordinate system. The 2D plane may include or correspond tovirtual object1712,1804 and the opening may include or correspond toopening1714,1806. The first portion may comprise an occluded portion of the third virtual object; and the second portion may comprise a non-occluded portion of the third virtual object.
• In a particular implementation,method1900 may include positioning, based on the coordinate system and based on the opening, an axis (e.g.,1808) in the virtual environment, determining a camera pose associated with the image capture device, and positioning a fourth virtual object in the virtual environment based on the camera pose. The camera pose may include or correspond to1816 and the fourth virtual object may include or correspond tovirtual object1810. The fourth virtual object may be positioned adjacent to the 2D plane, and the first portion of the third virtual object and the second portion of the third virtual object also be determined based on the fourth virtual object. Additionally, or alternatively,method1900 may include positioning, based on the opening, a fifth virtual object in the virtual environment. The fifth virtual object may include or correspond to thevirtual object1716,1810. The fifth virtual object may be positioned adjacent to the 2D plane and, in such implementations, the first portion of the third virtual object and the second portion of the third virtual object may also be determined based on the fifth virtual object.
• Thus,method1900 describes operation of a device (e.g.,110,150) to enable delivery of a multimedia message that corresponds to delivery of a package (e.g., an item). The multimedia message may adventurously display AR content and video content in a live video stream generated by a camera of the device. The multimedia content may present the video content in the camera stream, such that the video content realistically appears to be projected by or onto the first virtual object.
• The operations described with reference to the methods ofFIGS.12-14 and19 may be controlled by one or more a processing unit such as a central processing units (CPUs), controllers, field-programmable gate array (FPGA) devices, application-specific integrated circuits (ASICs), other hardware devices, firmware devices, or any combination thereof. It is noted that one or more operations described with reference to one of the methods ofFIGS.12-14 and19 may be combined with one or more operations of another ofFIGS.12-14 and19. For example, one or more operations ofmethod1200 may be combined with one or more operations ofmethod1300. As another example, one or more operations ofmethod1200 may be combined with one or more operations ofmethod1400. As a further example, one or more operations ofmethod1200 may be combined with one or more operations ofmethod1900.
• Although one or more of the disclosed figures may illustrate systems, apparatuses, methods, or a combination thereof, according to the teachings of the disclosure, the disclosure is not limited to these illustrated systems, apparatuses, methods, or a combination thereof. One or more functions or components of any of the disclosed figures as illustrated or described herein may be combined with one or more other portions of another function or component of the disclosed figures. Accordingly, no single implementation described herein should be construed as limiting and implementations of the disclosure may be suitably combined without departing from the teachings of the disclosure.
• Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, and composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein can be utilized according to the present examples. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

Claims (25)

1. A method comprising:

displaying, at a display of a mobile device, a first virtual object in a camera video stream generated based on an image capture device of the mobile device, the first virtual object associated with presentation of video content in the camera video stream; and

displaying, concurrently at the display and with the camera video stream, the first virtual object, the video content, and virtual light from a virtual light source associated with the first virtual object.

2. (canceled)

3. The method ofclaim 1, further comprising:

displaying a second virtual object in the camera video stream; and

wherein the first virtual object includes a film projector: projector; and

wherein:

the virtual light source is positioned at the first virtual object; or

the second virtual object comprises a projector screen, and the video content is presented on a surface of the projector screen.

4. The method ofclaim 1, wherein:

the first virtual object includes a three dimensional monitor; and

the virtual light source is positioned within the first virtual object; or

the video content is presented on a surface of the three dimensional monitor.

5-6. (canceled)

7. The method ofclaim 1, further comprising:

positioning, based on a coordinate system, a two dimensional (2D) plane in a virtual environment;

determining an opening in the 2D plane;

based at least on the 2D plane, the opening, or both, determining a first portion of a third virtual object and a second portion of the third virtual object; and

presenting the first portion of the third virtual object in the camera video stream based on a camera pose.

8-10. (canceled)

11. The method ofclaim 1, further comprising:

displaying a selectable icon concurrently with display of the camera video stream, the selectable icon configured to enable recording of the camera video stream including the first virtual object.

12. A device comprising:

a display;

a processor coupled to the display; and

a memory storing one or more instructions that, when executed by the processor, cause the processor to:

display, at the display, a first virtual object in a camera video stream, the first virtual object associated with presentation of video content in the camera video stream; and

displaying, concurrently at the display and with the camera video stream, the first virtual object, the video content, and virtual light from a virtual light source associated with the first virtual object.

13. The device ofclaim 12, further comprising:

a display an image capture device coupled to the processor; and

wherein the image capture device is configured to capture a field of view including an object and generate image data associated with the field of view, the camera video stream based on the image data.

14. The device ofclaim 13, wherein the one or more instructions, when executed by the processor, further cause the processor to initiate storage at the memory of the video content including the first virtual object and the video content presented via the display.

15-16. (canceled)

17. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

displaying a first virtual object in a camera video stream, the first virtual object associated with presentation of video content in the camera video stream; and

displaying, concurrently at the display and with the camera video stream, the first virtual object, the video content, and virtual light from a virtual light source associated with the first virtual object.

18. (canceled)

19. The computer-readable storage medium ofclaim 17, wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

providing a user interface via a display, the user interface provided concurrently with display of the camera video stream, the user interface configured to enable recording of the camera video stream including the video content.

20. The computer-readable storage medium ofclaim 17, wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

detecting a marker or an object in the camera video stream;

establishing a coordinate system based on the detected marker or the detected object, the coordinate system associated with a real world environment or a virtual environment;

positioning, based on the coordinate system, a two dimensional (2D) plane in a virtual environment;

determining an opening in the 2D plane;

determining, based at least on the 2D plane, the opening, or both, a first portion of a third virtual object and a second portion of the third virtual object; and

presenting the first portion of the third virtual object in the camera video stream based on the coordinate system.

21. The method ofclaim 1, further comprising:

generating, at the mobile device, the camera video stream, and

wherein the camera video stream includes an object, and

wherein the first virtual object displayed in the camera video stream on a surface of the object.

22. The method ofclaim 1, wherein the virtual light source configured to be activated and deactivated such that the virtual light appears to blink.

23. The method ofclaim 1, wherein the virtual light is intermittently displayed concurrently with the camera video stream during the presentation of the video content in the camera video stream.

24. The method ofclaim 1, further comprising:

positioning, based on a coordinate system, a two dimensional (2D) virtual object on a surface of an object included in the camera video stream; and

positioning a second virtual object adjacent to the 2D virtual object and in relation to a virtual axis that extends through an opening in the 2D virtual object.

25. The method ofclaim 24, further comprising:

presenting, based on a camera pose, a first portion of the first virtual object in the camera video stream based on the coordinate system such that a second portion of the first virtual object is occluded at least by the second virtual object.

26. The method ofclaim 1, wherein displaying a first virtual object includes displaying, based on a camera pose, a first portion of the first virtual object in the camera video stream such that a second portion of the first virtual object is occluded at least by a second virtual object that is positioned adjacent to a two dimensional (2D) virtual object and in relation to a virtual axis that extends through an opening in the 2D virtual object.

27. The method ofclaim 1, wherein the first virtual object is a first augmented reality (AR) object.

28. The method ofclaim 1, further comprising:

generating the camera video stream; and

displaying, at the display, the camera video stream in real-time.

29. The method ofclaim 1, further comprising:

during presentation of at least a portion of video content, generating a recording by recording an individual viewing the portion of the video content; and

transmitting the recording.

Images