BACKGROUNDThe scope of television programs, movies and other video assets available for consumption continues to expand. This expansion includes growth in the number and types of assets available, as well as growth in the number of ways a specific asset might be available. For example, a particular asset available from a data stream that is simultaneously transmitted to multiple destinations might also be available in the form of a unicast stream from a video on demand (VOD) server. With continued expansion of video asset availability, there remains a need for improved devices and techniques for navigating among available assets.
SUMMARYThis Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the invention.
In at least some embodiments graphical descriptor data associated with a particular video asset includes data for one or more graphical descriptor images. The video asset might be, e.g., a movie, a sporting event, a television show or another type of programming transmitted on a multicast basis. The graphical descriptor images can be, e.g., screen images, poster art or other types of images. The graphical descriptor data may also include graphical descriptor metadata related to each graphical descriptor image. The graphical descriptor images can be simultaneously presented on a display screen with the video asset content and used to navigate to different portions of content in that video asset based on the contents of the graphical descriptor metadata. The graphical descriptors can also be used to navigate to content of one or more different assets. In some embodiments, content data for a video asset and associated graphical descriptor data received at a destination device may also be transferred to a transferee device. The transferee device may then simultaneously present the content from the transferred data and the associated graphical descriptor images. A user of the transferee device may then select one of the graphical descriptor images to navigate to content in a different portion of the asset or to content of a different asset based on associated graphical descriptor metadata.
BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a block diagram showing elements in a network according to some embodiments.
FIG. 2 is a block diagram showing elements at a destination premises according to some embodiments.
FIG. 3 is another block diagram showing elements according to some embodiments.
FIGS. 4A through 4D illustrate one manner in which an exemplary transferee device may use data and metadata for an asset according to some embodiments.
FIGS. 5A and 5B are diagrams showing examples of data streams according to some embodiments.
FIG. 6 is a state diagram showing various operations performed by a destination device according to some embodiments.
FIG. 7 is a flow chart showing operations associated with transferring a copy of content data and associated graphical descriptor data according to some embodiments.
FIG. 8 is a state diagram showing various operations performed by a transferee device according to some embodiments.
DETAILED DESCRIPTION“Video content” generally refers to information displayed (or displayable) to a user in the form of one or more images. Video content may have associated audio content, i.e., information presented (or presentable) to a user in audio form. Video content can be communicated or stored in the form of data. Video content data providing a digital representation of a particular video content can be created using any of various encoding techniques. Such techniques include, but are not limited to, compression according to a Motion Picture Experts Group (MPEG) standard (e.g., MPEG-2), compression according to the ITU-T H.264 (ISO/IEC MPEG-4) advanced video coding (AVC) standard, etc. Video content data may be included in a single data stream that also includes associated audio content, or associated audio content data may be carried in a separate stream.
A video content data stream can be transmitted over a network by modulating some type of signal (e.g., an optical signal, an RF carrier signal, an electrical signal carried over a twisted pair) that is communicated over some type of medium (e.g., optical fiber, coaxial cable, twisted pair conductor, free space, etc.) using one or more of various types of communication protocols (e.g., internet protocol). The received signal may then be processed to extract the video content data stream. In addition to demodulating the received signal, such extraction may include demultiplexing by isolating a signal carried on a particular optical wavelength or RF frequency from signals on other wavelengths or frequencies, by isolating certain data segments from other data segments, and/or by other types of demultiplexing techniques. Once the data stream has been extracted, data from that stream can then be decoded and used to generate appropriate electrical signals. Those electrical signals can then be output to a display device (e.g., a television) so as to cause the display device to present the video content on a display screen. Video content data can also be stored in some type of storage device (e.g., a magnetic disk drive) and then later retrieved for decoding and presentation in a similar manner.
A collection of video content (e.g., a series of image frames) may be treated as a unit, or a “video asset,” for some purposes. In some cases, for example, a video asset may refer to a collection of video content that a user would normally perceive as a single item of content. Examples of such video assets include a movie, an episode of a television series, coverage of a particular sporting event, a news program, coverage of a concert or other event, etc. A video asset may also include associated audio content (e.g., a soundtrack) and other associated information (e.g., graphical descriptors as discussed below). A “video asset data stream” may be, for example, a series (or other group) of protocol data units (PDUs) that contain (or that are identified as possibly containing) video content data for a particular asset. The PDUs of a video asset data stream may also contain other data associated with the video asset (e.g., audio content data, text for closed captioning, etc.), or such other data could be carried in separate streams. PDUs of a video asset data stream may be time-multiplexed with PDUs of other data streams for transmission. Those other data streams could contain data for associated audio content, other associated data, data for other video assets, or other kinds of data. Examples of video asset data streams are discussed below in connection withFIGS. 5A and 5B. A video asset data stream might be periodically interrupted for reasons unrelated to multiplexing. For example, a stream containing content data for a particular asset might be periodically interrupted and replaced, in whole or in part, with a stream containing content data for a different asset (e.g., a commercial).
In some embodiments, a “graphical descriptor” is an image usable to describe a video asset or a portion of a video asset. “Graphical descriptor image data” refers to data that represents a graphical descriptor image for purposes of communication or storage, and can be created using any of various encoding techniques. Examples of such techniques include, but are not limited to, encoding and compression according one or more Joint Photographic Expert Group (JPEG) standards, creation of a raw bitmap, etc. A graphical descriptor may also have related graphical descriptor metadata. Such metadata can include additional information used to process a graphical descriptor image or a user selection of such an image. Such metadata may also include text that can be presented in conjunction with a graphical descriptor on a display screen, as well as code (or other data) that can cause various actions if a graphical descriptor is selected or otherwise acted upon. Graphical descriptor image data and the related graphical descriptor metadata for a particular graphical descriptor can be collectively referred to as “graphical descriptor data.” A graphical descriptor “associated” with a video asset may describe that same video asset or a portion thereof, or it may describe a different video asset (or portion thereof).
For convenience, certain embodiments are initially described by example of a network that delivers video content to a Set Top Terminal (STT) destination device in a hybrid fiber coaxial (HFC) access sub-network, with data for that video content communicated in one or more MPEG-2 transport streams. This initial example is not a limitation. Other embodiments include systems and methods in which content is delivered to other types of destination devices (e.g., a display device, a smart phone, or a general purpose computer). Similarly, additional embodiments include systems and methods that employ other types of communication media (e.g., a fiber to the home (FTTH) passive optical network (PON)), satellite, wireless, and/or other communication protocols (e.g., Internet Protocol (IP)).
FIG. 1 is a block diagram showing certain elements of acommunication network10 according to some embodiments. Afirst media source11 outputs multicast video asset data streams. In particular, the data streams output bysource11 contain data for video assets that are intended for simultaneous delivery to numerous destination devices throughoutnetwork10. The data streams fromsource11 may be forwarded directly to destination devices, or may be aggregated with video asset data streams from other sources before ultimate delivery to destination devices. In addition, intermediate network elements may receive video asset data streams emanating fromsource11 and reformat those streams according to a different communication protocol.Media source11 may be a media server, a collection of media servers, a network media hub that receives feeds from other sources (e.g., satellite downlinks) and creates video asset data streams, or some other network element or collection of elements. Although not shown inFIG. 1,source11 includes processing devices, memory, and communication devices (e.g., network interface hardware) for carrying out operations ofsource11 described herein.
Also shown inFIG. 1 is asecond media source12 that outputs unicast video asset data streams. In other words, each of the data streams output bysource12 contains data for a video asset that is typically intended for delivery to a single destination device.Source12 may be, e.g., an enhanced video on demand (VOD) server, or collection of servers, that provide(s) various of types of video asset data as described herein.Source12 could include, e.g., Internet video servers, servers outputting content from provider vaults, etc.Source12 may include processing devices, memories, and communication devices (e.g., network interface hardware) for carrying out operations ofsource12 described herein.
Sources11 and12 communicate data streams over anetwork backbone13.Backbone13 may include national and/or regional links, routers, additional media sources, etc. Data streams communicated bysources11 and12 overbackbone13 may be received at a central office14 (e.g., a head-end) of an access sub-network15 (e.g., an HFC, fiber optic, or wireless access sub-network). Those data streams, together with data streams from other sources (not shown) are multiplexed, modulated and transmitted overaccess sub-network15 to adestination device17 located at a destination premises16 (e.g., a user's home, a business or an institution).Destination device17 may demodulate the signals received overaccess sub-network15 and extract a video asset data stream for an asset selected by a user. As will be described in more detail in connection withFIG. 2,destination device17 may then decode that data and generate signals for output of the selected asset on adisplay device18, for example.Display device18 anddestination device17 may be incorporated into a single device, or each may be a stand-alone unit.Destination device17 may also store received data in a memory, and/or transfer a copy of that data to atransferee device19.
For simplicity,FIG. 1 only shows twomedia sources11 and12, asingle access sub-network15, and asingle destination premises16 having asingle destination device17.Network10 may include numerous additional media sources. Moreover,sources11 and12 could be in a single physical location. For example,sources11 and12 could be implemented as different software routines executing on a single computer.Access sub-network15 may serve numerous additional destination premises throughout a particular region, and one or more of those premises may include multiple destination devices capable of simultaneously receiving and/or transmitting signals insub-network15.Network10 may also include numerous additional access sub-networks, with each of those sub-networks also serving numerous destination devices. Various routers and other network elements may be located inbackbone13 or elsewhere innetwork10. Because the operation of such devices is known or would be readily apparent to a person of ordinary skill in light of the disclosures herein, such devices are not shown inFIG. 1 or further discussed.
FIG. 2 is a block diagram showing additional details of devices atdestination premises16, as well as display of a video asset and associated graphical descriptors, according to some embodiments. In the example ofFIG. 2, devices atdestination premises16 include anSTT destination device17, atransferee device19, atelevision display device18, and aremote control handset20. As discussed below, other types of devices could operate as destination device, as a transferee device, and as a display device.
Destination device17 communicates withnetwork10 oversub-network15 and includes an interface (I/F)25 that provides a physical connection to the communication medium ofaccess sub-network15. If for example, theaccess sub-network15 is an HFC network,interface25 may provide an RF interface for communication over coaxial cable. Ifaccess sub-network15 is a Fiber To The Premises (FTTP) Passive Optical Network (PON),interface25 may provide an optical interface for communication over an optical fiber. Ifaccess sub-network15 is a wireless network,interface25 may provide an RF interface for communication over free space. Ifaccess sub-network15 is a Digital Subscriber Line (DSL) network,interface25 could include a DSL modem.Interface15 could also include a data connection (e.g., an Ethernet port) that permitsdestination device17 to communicate with a separate device (e.g., an optical network terminal (ONT), a separate modem, a separate transceiver) that communicates overaccess sub-network15.Interface25 can include hardware for tuning to different channels (e.g., RF frequency channels, optical wavelengths), for demodulating signals received in those tuned channels, and for outputting data from demodulated signals for further processing.Interface25 also may include components for modulating data onto an upstream signal and transmitting that signal tonetwork10.
Destination device17 further may includememory26 for storing instructions and data and aprocessor27 for executing instructions and controlling operation ofdevice17. Although a single block is shown formemory26 and a single block shown forprocessor27, memory and operations ofdevice17 could respectively be distributed across multiple memory devices and multiple processors located withindevice17. For example,device17 may include additional processors for executing video and audio CODEC (compressor/decompressor) routines, etc.Memory26 may include volatile and non-volatile memory and can include any of various types of storage technology, including but not limited to read only memory (ROM) modules, random access memory (RAM) modules, magnetic tape, magnetic discs (e.g., a fixed hard disk drive or a removable floppy disk), optical disk (e.g., a CD-ROM disc, a CD-RW disc, a DVD disc), flash memory, and EEPROM memory.Processor27 may be implemented with any of numerous types of devices, including but not limited to general purpose microprocessors, application specific integrated circuits, field programmable gate arrays, and combinations thereof. In at least some embodiments,processor27 carries out operations ofdestination device17 described herein according to machine readable instructions stored inmemory26 and/or stored as hardwired logic gates withinprocessor27.Processor27 may communicate with and controlsmemory26 andinterface25 over one ormore buses28.Device17 also may include one or more audio and/or video interfaces29-1 through29-n(e.g., left and right audio channel outputs, a video output, an HDMI output) over which signals are output for presentation of video content ontelevision18 and presentation of audio content on one or more speakers (not shown). An interface30 (e.g., an infrared or RF interface) receives input fromremote control handset20. Other types of input devices (e.g., a computer mouse, a gesture-responsive controller) could also be supported bydevice17.
Processor27 may also communicate with peripheral interfaces31-1 through31-noverbus28.Device17 may use peripheral interfaces31-1 through31-nto communicate with other devices (such as, but not limited to, transferee device19). Each of interfaces31-1 through31-nmay include appropriate hardware and stored instructions for sending and receiving communications according to a defined standard and for passing such communicated data to and fromprocessor27 and/ormemory26. In the example ofFIG. 2, interface31-1 is a Universal Serial Bus (USB) interface, interface31-2 is an IEEE 802.11 WiFi interface, interface31-3 is an Ethernet interface, and interface31-nis a Multimedia over Coax Alliance (MOCA) interface. Other types of peripheral communication interfaces could be employed.Processor27 also controls operation of interfaces31-1 through31-n.
Memory26 ofdevice17 can be used to store instructions and data used bydevice17 to carry out conventional operations such as decoding content data, generating VOD commands and other information for communication to elements innetwork10, providing an electronic program guide (EPG), etc.Memory26 can also be used to store video asset data for later playback (retrieval and decoding to view content encoded by such data). In addition to data used for conventional operations,memory26 may also store graphical descriptor data.Processor27 decodes graphical descriptor image data for display ontelevision18 or another display device and performs operations based on related graphical descriptor metadata, as discussed below.
Data storage and processing operations described herein fordevice17 could also be distributed across memory and processing components located in other devices (e.g., a computer, a second STT, a transferee device such asdevice19, etc.). Such other device(s) could be in communication withdevice17 in a local network or could be in communication withdevice17 vianetwork10.
Transferee device19 can communicate withdestination device17 over one of interfaces31-1 through31-nto download content data and related graphical descriptor data, as also discussed below. In the embodiment ofFIG. 2,device19 may be a “smart” phone or other mobile communication device and that includes memory, processing, display and other components and capabilities such as are described herein fordevice19.Device19 includes aninterface37 that corresponds to the one of interfaces31-1 through31-nover whichdevice19 communicates withdevice17.Device19 can useinterface37 to receive data transferred frommemory26 ofdevice17.Device19 also includes atransceiver38 used for direct communication over a wide area wireless network (e.g., a 3G network).Transferee device19 could also (or alternatively) communicate with device17 (e.g., to exchange video asset and graphical descriptor data) via that wide area network,network10 andinterface25. Aprocessor39 may also be configured to execute instructions so as to perform various operations ofdevice19 as described herein and to control operation of other components ofdevice19. Such instructions may be stored inmemory40 as executable instructions and/or as hard wired logic withinprocessor39. For example, stored inmemory40 are one or more applications that configureprocessor39 to communicate withdevice17 usinginterface37, to decode content data and display video content ondisplay screen41 and output audio content onspeaker42, to display graphical descriptors and perform operations based on related graphical descriptor metadata, etc.Device19 further includes amicrophone43,keypad44 andbattery45.
In the example ofFIG. 2,device17 is receiving numerous multicast data streams containing data for various video assets. Each of those streams could, e.g., carry data for a particular service such as HBO, ESPN, etc., with each service being assigned a virtual channel number that a user can select to begin watching the asset currently carried on a selected service. One of the streams received bydevice17 may be multicast videoasset data stream50, which is currently providing content data forvideo asset51. The example ofFIG. 2 further assumes that a user ofdevice17 andtelevision18 has selectedvideo asset51 for viewing by, e.g., providing input withhandset20 selecting a virtual channel corresponding to stream50. For simplicity,FIG. 2 does not show other multicast streams that may be communicated todevice17 within the samesignal communicating stream50.Stream50 also includes graphical descriptor data associated withasset51.
Asmulticast stream50 is received,device17 extracts data fromstream50, decodes extracted video content data corresponding toasset51 and generates corresponding signals. Those signals are then output totelevision18 over interface29-nso as to permittelevision18 to display content ofvideo asset51.Device17 may also store extracted graphical descriptor data fromstream50 inmemory26. In response to a user input fromhandset20 while content forasset51 is being displayed,device17 may decode the graphical descriptor image data and cause presentation of the corresponding graphical descriptors in conjunction withasset51 content on a display screen oftelevision18. In the example ofFIG. 2,device17 has caused such graphical descriptors52-55 to be displayed at the bottom of the screen. Audio data associated withasset51 may also be extracted and decoded, and signals generated and sent to a device (e.g.,device18, separate speakers, headphones) for output of the audio associated withasset51.
Each of graphical descriptors52-55 may be an image that provides a graphical description of a different portion of content inasset51, which a user can select to navigate to that content portion. In the present example,asset51 is a movie having a run-time (duration) of T minutes. In other words, ifasset51 is continuously displayed without interruption and at the proper rate, the movie would begin at time 0 and end T minutes later. In the present example,graphical descriptor52 describes a portion ofasset51 that commences0.2T minutes after theasset51, e.g., a movie, begins.Descriptors53,54 and55 respectively describe portions ofasset51 that commence 0.4T, 0.6T and 0.8T minutes after the movie begins.
The image in a particular descriptor can be an actual frame taken directly from the described portion ofasset51, as shown withgraphical descriptor52. This need not be the case, however. As one example, and as shown bydescriptor54, an image that combines elements from multiple frames within a described portion ofasset51 may provide a more desirable description than any individual frame. As another example, and as shown bydescriptor53, a specially-prepared still image of an actor may show an important character from a scene better than any actual frame of the asset. As yet another example, and as shown bydescriptor55, specially prepared artwork may symbolically represent events in a portion of the content. Other types of graphical descriptors could also (or alternatively) be used. A graphical descriptor and/or its related metadata may also be linked to a particular frame or sequence of frames in an asset in such a manner as to cause the graphical descriptor to only be displayed during that particular frame or sequence of frames.
With regard to each of graphical descriptors52-55, the related graphical descriptor metadata stored inmemory27 may include pointer data that identifies the described portion ofasset51. For example, stored metadata relating todescriptor52 may include an identifier ofasset51 and a time index ofasset51 corresponding to 0.2T minutes from the asset beginning. As further explained below, this metadata can be used (upon selection of descriptor52) to cause display of the part ofasset51 that begins at time 0.2T. The stored metadata relating to descriptors53-55 similarly may include anasset51 identifier and pointers (e.g., time indices) corresponding to relevant portions ofasset51. The metadata related to each graphical descriptor may also include other types of data. For example, the metadata relating to a descriptor could include a synopsis or other textual description of a scene or other content portion that corresponds to that descriptor. The textual description could be displayed in conjunction with the graphical descriptor (either automatically or in response to an additional user input). In some embodiments, the graphical descriptor metadata relating to each graphical descriptor may include an address innetwork10 forunicast source12 and/or for other elements innetwork10.
[33] After graphical descriptors52-55 are displayed, a user may select one of the descriptors to indicate a desire to skip ahead (or back) to a particular portion ofasset51. For example, and as shown inFIG. 2, a user can highlightgraphical descriptor54 withcursor56 by, for example, pressing arrow buttons (not shown) onhandset20. After highlightinggraphical descriptor54, the user may provide an input onhandset20 indicating selection of theasset51 portion corresponding todescriptor54. Upon receiving data indicative of this input,processor27 ofdevice17 may generate a request message that includes theasset51 identifier and the time index included in the metadata related tographical descriptor54.Device17 may then transmit that request message upstream tounicast source12 or to another appropriate element innetwork10. In response to the request message from device17 (or in response to a message from another element resulting from thedevice17 request message),unicast source12 begins transmitting a unicast video asset data stream todevice17 that includes content data forasset51. However, the content data in that stream fromsource12 begins at a point corresponding to the time index included in the request message.
FIG. 3 is a blockdiagram showing device17 andtelevision18 aftersource12 begins transmitting a unicast videoasset data stream60 in response to the request fromdevice17. A unicast video asset data stream may be transmitted according to either a push or a pull model. For convenience,device19 is omitted fromFIG. 3. The data inunicast stream60 contains data forasset51 beginning at the time index taken from the graphical descriptor metadata related to selected graphical descriptor54 (FIG. 2) and included in the request message sent bydevice17. Upon selection ofgraphical descriptor54 by the user,device17 may stop decoding data frommulticast stream50. Upon receipt of thesignal containing stream60,device17 begins, for example, extractingstream60, decoding data fromunicast stream60, generating electrical signals using that decoded data, and outputting those electrical signals over AN interface29-n.As a result, and as also shown inFIG. 3,television18 starts to display content ofasset51 commencing 0.6T minutes after the beginning ofasset51.
Unicast stream60 may also contain graphical descriptor data. If so,device17 may store the graphical descriptor data instream60 and process that graphical descriptor data in a manner similar to the graphical descriptor data instream50. Graphical descriptor data instream60 may be the same or different than the graphical descriptor data instream50. If a unicast session is delivered in response to selection of a descriptor in a previously-delivered multicast session, for example, the unicast session could be augmented with additional graphical descriptor data during a post-production process. Such post-production could occur atdevice17. A user viewing a unicast stream might also be able to create personal descriptive data (e.g., screen shots from favorite scenes) and store that personal descriptive data for use with future viewings of the asset. Such personal descriptive data could also be shared with friends, etc.
As further shown inFIG. 3,device17 continues to receive a signal containing multicastvideo asset stream50. However,device17 may simply ignoremulticast stream50 in the same manner as it may ignore numerous other multicast streams carried overaccess sub-network15 at any given time. Alternatively,device17 may store content data and associated graphical descriptor data frommulticast stream50 while simultaneously decoding content data from unicast stream60 (and while storing graphical descriptor data fromstream60, if present). As but one possible scenario, a first user may have initially configureddevice17 to store data fromstream50 so thatasset51 could be played back at a later date. Whiledevice17 is recordingasset51 frommulticast stream50, a second user might then turn ontelevision18 andselect multicast stream50 for viewing (e.g., by providing input todevice17 indicating a virtual channel corresponding to multicast stream50). The second user might then decide to begin watchingasset51 starting at a different portion of the asset. For example, the second user may have selected themulticast stream51 virtual channel midway into the runtime ofasset51 and wish to begin watching from the beginning. However, the first user might also wish to continue recordingasset51 frommulticast stream50 so thatasset51 will be available for playback frommemory27 at the first user's convenience. The second user, who will be watching a unicast version ofasset51, may or may not see the same advertisements as are shown in the multicast version ofasset51 being recorded bydevice17. Moreover, the version ofasset51 being recorded bydevice17 could differ from what a user would see ifasset51 were viewed in real-time asstream50 is received. For example,stream50 could include additional data that is automatically stored bydevice17 in response to a user command to storeasset51 for later viewing, but which is not extracted fromstream50 ifasset51 is viewed in real time. This additional data could include longer versions of commercials, background video ofasset51, etc.
In the example thus far, and as shown inFIG. 2, graphical descriptors52-55 associated withvideo asset51 describe portions ofasset51 that commence at different points in the run-time ofasset51. This need not be the case, however. In other embodiments, one or more of the graphical descriptors associated with a video asset may describe all or part of a different video asset. As but one example, a video asset may be a movie or television program starring a particular actor. Each of multiple graphical descriptors associated with that asset may be a miniature version of a movie poster or other artwork describing a different movie starring that same actor. Each of those descriptors may in turn have related metadata that contains an identifier and/or a textual synopsis of the movie represented by the descriptor. Upon receipt of user input selecting of one of the descriptors,device17 could then generate and transmit a request message that causes source12 (FIG. 1) to begin sending device17 a unicast video asset data stream that includes the content data for the identified movie. As another example, each of multiple graphical descriptors associated with a movie video asset may be a miniature version of a poster for another movie in the same genre.
Combinations of various types of graphical descriptors (and related metadata) can be associated with a particular video asset. In one embodiment, a video asset of genre A starring actor B is associated with multiple sets of graphical descriptors. A first of those sets may include graphical descriptors, similar to those ofFIG. 2, that describe different portions of the video asset. A second of those sets may include graphical descriptors that describe other assets in genre A. A third of those sets may include graphical descriptors that describe other assets starring actor B. In response to data indicative of a first input from a user,device17 may cause a display of a menu (e.g., on television18) that allows the user to select a submenu corresponding to one of the graphical descriptor sets. In response to data indicative of a second input selecting a submenu,device17 then causes display of the graphical descriptors corresponding to that submenu. Upon receipt of data indicative of another user input selecting a displayed descriptor,device17 generates a request message that includes an asset identifier (or an asset identifier and a time index) from the metadata related to the selected descriptor. After sending that request message upstream,device17 begins receiving content data for the requested video asset (or asset portion).
A video asset corresponding to a particular graphical descriptor may be available by multicast. For example, each of one or more graphical descriptors could refer to content that is currently available in another multicast data stream. Related graphical descriptor metadata could identify the appropriate multicast stream. In response to data indicative of a user selection of a descriptor referring to content in another multicast stream,processor27 could generate an internal request that causes extraction of content data (and graphical descriptor data) from the other multicast stream. Under some circumstances, this may occur in order to conserve bandwidth, for example.
Returning toFIG. 2,device19 can communicate withdevice17 via one of interfaces31-1 through31-n.In some embodiments,device17 can store content data and associated graphical descriptor data inmemory26. That data (or a copy of that data) can then be transferred todevice19, the transferee device in this example, and stored inmemory40.Device17 might alternatively transfer such data totransferee device19 without interim storage on device17 (or with only incidental buffering or storage ondevice17 prior to transfer). Subsequent to receiving and storing the transferred data,device19 can retrieve content data frommemory40, decode that content data, and cause display of the video asset ondisplay screen41.Device19 may also decode graphical descriptor image data stored inmemory40 and generate displays of the graphical descriptors in conjunction with the video asset. In response to a user selection of a graphical descriptor,device19 can then take appropriate action (e.g., begin displaying the video asset from a point corresponding to the graphical descriptor). In some embodiments,device19 may include one or more application programs stored inmemory40 that permitprocessor39 to decode and process content data, graphical descriptor image data and metadata in the form received from network10 (FIG. 1). In such an embodiment,device17 can simply store such data inmemory26 as it is received, and then transfer that data tomemory40 ofdevice19 without significant modification. In other embodiments,device17 may modify data received fromnetwork10 prior to transferring such data todevice19. For example, data received bydevice17 fromnetwork10 may be encrypted, anddevice17 may decrypt that data. As another example,device17 may separate the data for the video (and audio) content of an asset from the associated graphical descriptor data.Device17 may then transcode the content data by decoding it and then recoding it using a different type of CODEC.Device17 may also modify the content data for a particular transferee device, e.g., to better suit a smaller display screen or a display screen with a different aspect ratio. The graphical descriptor metadata related to graphical descriptor image data may be converted to a different form (e.g., XML) and the graphical descriptor image data marked to accommodate any conversion of the related metadata.
FIGS. 4A-4D illustrate one manner in whichtransferee device19 may use content data and graphical descriptor data for a video asset after transfer of such data, e.g., tomemory40 fromdevice17. At some time subsequent to storing such data forasset51 inmemory40 ofdevice19, a user may provide an input (e.g., via keypad44) indicating a desire to viewasset51. In response, and as shown inFIG. 4A,processor39 ofdevice19 begins retrieving and decoding content data stored inmemory40 and causes the corresponding content ofasset51 to be displayed onscreen41. Subsequently, the user provides another input indicating a desire to access a menu providing options for displaying different portions ofasset51. In response, and as shown inFIG. 4B,processor39 ofdevice19 decodes graphical descriptor image data, stored inmemory40, for example, and causes display of graphical descriptors52-55 ondisplay41. The user next provides an input highlighting and selecting descriptor54 (FIG. 4C). In response,processor39 determines a time index from graphical descriptor metadata related todescriptor54 and stored inmemory39.Processor40 then begins retrieving and decoding content data for the portion ofasset51 identified by that time index and causes display of the corresponding content on display screen41 (FIG. 4D).
Device19 may also be configured to display graphical descriptors, in conjunction with playback of a first asset from content data inmemory40, that describe different assets and that have related metadata pointing to those different assets. In response to selection of such a descriptor,device19 could take various types of action. If a descriptor selected during first asset playback has related metadata pointing to a different asset for which content data is also stored inmemory40 ofdevice19,processor39 could stop playback of the first asset and begin playback of the different asset. If a selected descriptor has related metadata pointing to a different asset for which content data is not stored inmemory40,processor39 may prompt the user for authorization to download the differentasset using transceiver38. As another possibility,processor39 may prompt the user for authorization to cause source12 (or another source in network10) to transmit the content todevice17 for recording inmemory26. If the user ofdevice19 provides authorizing input,processor39 can send one or more messages to elements innetwork10 usingtransceiver38. In response to those messages, an element innetwork10 can begin transmission of a data stream containing content for the desired video asset, and a signal can be sent todevice17instructing device17 to record that data stream. After the content data for the desired asset has been stored inmemory26 ofdevice17, a copy of that data can be transferred tomemory40 ofdevice19 by connecting device19 (using interface37) and device17 (using one of interfaces31).
In some embodiments,device17 can also operate in a manner similar to that described fortransferee device19. For example,device17 may retrieve and decode video asset data and associated graphical descriptor data from local storage inmemory26 and cause display of the asset and its associated graphical descriptors. When a user selects one of those displayed graphical descriptors,device17 may then cause a jump to a portion of the asset identified by a time index associated with the selected descriptor by retrieving and decoding a different portion of the asset data frommemory26. As another example, selection of a graphical descriptor while replaying content data frommemory26 could causedevice17 to begin decoding and outputting video for a different asset stored inmemory26, or may causedevice17 to transmit a message to initiate transmission of a stream containing a different asset.
FIG. 5A is a diagram showing four packets101-104 of a videoasset data stream100 according to at least one embodiment. In the example embodiment ofFIG. 5A,stream100 is an MPEG-2 transport stream carrying the video content data for a particular asset. As with conventional MPEG-2 transport streams, each of the packets instream100 includes afield110 that contains a Packet IDentifier (“PID”) used by a receiving device to isolatestream100 from numerous other streams that might be multiplexed together. In a conventional MPEG-2 transport stream carrying video content data, a packet contains a plurality of variable-length Packetized Elementary Stream (PES) blocks containing data that encodes the video content. Unlike conventional MPEG-2 transport streams, however, certain PES blocks in certain packets ofstream100 are designated as graphical descriptor data blocks. Those graphical descriptor data blocks are then used to hold graphical descriptor data associated with the video asset data in other PES blocks ofstream100 packets. Graphical descriptor data blocks are shownFIG. 5A with cross-hatching. Specifically, the first PES block111 ofpacket101 and the first PES block112 ofpacket104 are graphical descriptor data blocks.Blocks111 and112, as well as other graphical descriptor data blocks instream100, are marked in an appropriate manner to indicate the contents of the block. In some embodiments, one or more fields in a PES block header can hold a value indicating that the block is a graphical descriptor data block and not a conventional PES block.
Although the example ofFIG. 5A shows graphical descriptor data blocks as the first PES blocks in packets, this need not be the case. Indeed, and because a graphical descriptor data block is marked to distinguish it from other PES blocks, graphical descriptor data blocks can be located in different positions from packet to packet. Moreover, more than one PES block in a packet could be used as a graphical descriptor data block.
As can be seen inFIG. 5A, every packet in a video asset data stream need not contain a graphical descriptor data block. In at least some embodiments, the image data for a single graphical descriptor and its related metadata may be only two to four kilobytes in size. This is very small relative to the size of the video content data for most video assets. This can permit transmission of all graphical descriptor data associated with a particular video asset in a negligible amount of time. For example, all graphical descriptor data associated with a movie video asset could be transmitted in the first few seconds of the movie. In some embodiments, however, the graphical descriptor data associated with an asset are repeatedly transmitted (e.g., every five minutes). For example, a user ofdevice17 may begin viewing an asset after it has commenced, and thusdevice17 may not have stored graphical descriptor data that was transmitted at the start of the asset. However, that graphical descriptor data can be stored when it is received in a repeat transmission occurring after the user has started viewing the asset. Thus, even if a user did not view a previously-transmitted portion of an asset when it was initially transmitted as part of a multicast stream, the user can jump back to that previously-transmitted portion by selecting a graphical descriptor corresponding to retransmitted graphical descriptor data. In response, a unicast transmission of that previously-transmitted asset portion can be initiated, as described herein, for example.
FIG. 5B is a diagram showing communication of graphical descriptor data according to another embodiment. In the example ofFIG. 5B, video content data for an asset is carried in packets of a first MPEG-2transport stream150. Four blocks151-154 ofstream150 are shown. As with other packets instream150, each of blocks151-154 may include afield160 that contains a PID for the video asset data stream. Graphical descriptor data are carried in packets of aseparate data stream170. Twopackets171 and172 ofstream170 are shown. Each packet instream170 includes afield180 that contains a PID that identifies packets as carrying graphical descriptor data associated with the video asset ofstream150. Included in each packet ofstream170 are one or more graphical descriptor data blocks (181,182). For example, a program map table carried in a different stream (not shown) may inform destination devices of the PIDs forstreams150 and170.
As with the embodiment ofFIG. 5A, the image data for a single graphical descriptor and its related metadata in the embodiment ofFIG. 5B is of very small size relative to the size of the video content data for most video assets. This permits communication of all graphical descriptor data associated with a particular asset in a relatively short group ofstream170 packets. Such a packet group can be communicated to and processed within a destination device in a few seconds. In some embodiments, a group of packets containing all graphical descriptor data associated with an asset is communicated instream170 when the first packets ofstream150 are communicated. At periodic intervals (e.g., every few minutes), another group of packets containing the same graphical descriptor data could be sent.
When a group of blocks containing all graphical descriptor data for an asset can be communicated quickly relative to the runtime of the associated video asset, and at least with regard to repeat transmission of such graphical descriptor data block groups during the asset runtime, there is no need to synchronize graphical descriptor data block transmission with a specific portion of an associated video asset.
FIGS. 5A and 5B only represent some examples of techniques by which graphical descriptor data and related graphical descriptor metadata associated with a video asset can be communicated in an MPEG-2 transport stream. Other embodiments utilize MPEG-2 transport streams in different manners. Still other embodiments do not use MPEG-2 transport streams at all. For example, content data for a video asset and associated graphical descriptor data can be communicated in a single stream using internet protocol (IP) frames.
Graphical descriptor data can be formatted so as to accommodate a particular type of application program within a destination device. In some embodiments, for example, a destination device can include one or more EBIF (Enhanced TV Binary Interchange Format) applications that process graphical descriptor image data and related graphical descriptor metadata. As another example, a destination device can include one or more OCAP (OpenCable Application Platform) applications that process graphical descriptor data.
FIG. 6 is a state and flow diagram showing various operations that may be performed bydestination device17 according to some embodiments. As shown inblock201,device17 receives a signal and extracts a data stream from that signal. The extracted data stream may contain video content data for an asset that has been selected by user, as well as graphical descriptor data associated with the selected asset.Device17 decodes the extracted video content data and generates electrical signals, for output totelevision18 or another display device, to cause display of the selected asset content.Device17 may also store the graphical descriptor data inmemory26. As indicated above, all graphical descriptor data associated with a particular asset may be transmittable in a short period of time. Accordingly,device17 need not continuously store such graphical descriptor data. After a complete set of the graphical descriptor data is received,device17 could simply ignore any retransmission of that same graphical descriptor data.Device17 need not perform all of the functions shown in block201 (or other blocks), and may selectively perform some tasks but not others.
The data stream extracted bydevice17 inblock201 may be a multicast stream (such asstream50 inFIG. 2) or may be a unicast stream. In at least some embodiments, video assets available from a VOD server or other type of unicast source also have associated graphical descriptor data.
In response to data indicative of a user input (e.g., fromhandset20 shown inFIG. 2), and as shown by path202,device17 transitions to the state of block203. In this state,device17 continues to decode video content data and output corresponding electrical signals totelevision18. However,device17 also may decode graphical descriptor image data and cause display of graphical descriptors in conjunction with content of the selected asset.Device17 may further cause display of textual information with one or more graphical descriptors based on information (e.g., synopses) contained in related graphical descriptor metadata. While in the block203 state,device17 may cause display of one or more submenus having different sets of graphical descriptors.
If the user decides not to select a displayed graphical indicator, and as shown bypath204, data indicative of an appropriate user input (e.g., pressing of a “cancel” or “exit” button on handset20) returnsdevice17 to theblock201 state. If the user instead provides an input selecting one of the displayed graphical indicators,device17 may transition on path205 to the state of block206. In the block206 state,device17 stops decoding video content data for the previously selected asset. Ifdevice17 was recording that asset, however, the recording may continue.Device17 may also generate a request message that contains an asset identifier from the metadata related to the selected graphical indicator. That identified asset may be the same as the previously selected asset or may be a different asset. If the identified asset is the same as the previously selected asset, the request message may also include a pointer (e.g., a time index) to a particular portion of the asset.Device17 may then send the generated request message to source12 or to another appropriate element innetwork10.
In response to the request message,source12 may begin transmitting a unicast data stream containing content data for the asset identified in the request message. Upon detecting that unicast stream, and as shown bypath207,device17 returns to theblock201 state and begins performing the operations of that state on the received unicast data stream.
Device17 may perform similar operations when playing back content previously stored in thememory26 ofdevice17. In theblock201 and block203 states, for example,device201 may simply retrieve video content data for the selected asset frommemory26 instead of extracting that data from a signal received overaccess sub-network15.Device17 may also perform similar operations if video content data for a selected asset were carried in one stream and associated graphical descriptor data were carried in a separate stream. In theblock201 and block203 states,device17 may simply extract data from both of those streams. If a selected graphical indicator corresponds to an asset for which data is being carried in another multicast stream, the request generated in the block206 state may be an internal request that causesdevice17 to begin extracting content data and graphical descriptor data from that other multicast stream, withpath207 then representing initial detection byprocessor27 of such extraction.
FIG. 7 is a flow chart showing operations that may be performed bydestination device17 when transferring a copy of content data and associated graphical descriptor data todevice19 or to another transferee device. Inblock230,device17 receives a request from device19 (via one of interfaces31) requesting download of a particular asset. Inblock231,device17 performs any necessary conversion of the content data and associated graphical descriptor data corresponding to the requested asset. This conversion can include conversion of EBIF, OCAP or other application data and/or related signaling. In some embodiments, some or all of the operations ofblock231 may alternately have been performed previous to block230, for example, in anticipation of a future request for the corresponding asset. In still other embodiments (e.g., where a transferee device anddevice17 can process data in the same format), some or all of the operations ofblock231 may not be performed. Inblock232, content data and associated metadata for the requested video asset are transferred todevice19.
FIG. 8 is a state diagram showing various operations that may be performed bytransferee device19 according to some embodiments.FIG. 8 assumes that content data and associated graphical descriptor data for a first asset have previously been stored in memory40 (or another storage) ofdevice19.FIG. 8 further assumes that a user ofdevice19 has previously provided input (e.g., via keypad44) selecting the first asset for presentation ondisplay screen41. In the state ofblock250,processor39 may retrieve content data for the first asset frommemory40, decode that content data, and cause presentation of the corresponding first asset content ondisplay screen41. In response to another user input, and as shown bypath251,device19 transitions to the state of block253.
In the block253 state,device19 may continue to decode video content data for the first asset and continue to present corresponding first asset content ondisplay screen41. However,device19 may also decode graphical descriptor image data and cause display of graphical descriptors in conjunction with content of the first asset.Device19 may further cause display of textual information with one or more graphical descriptors based on information (e.g., synopses) contained in related graphical descriptor metadata. While in the block253 state,device19 may cause display of one or more submenus having different sets of graphical descriptors.
If the user decides not to select a displayed graphical indicator, and as shown bypath254, an appropriate user input (e.g., pressing a key of keypad44), for example, may returndevice19 to theblock250 state. If the user instead provides an input selecting one of the displayed graphical indicators,device19 may determine if the selected graphical indicator describes the first asset (currently being presented on screen41) or a different asset. If the selected descriptor corresponds to the first asset, and as shown bypath255,device19 may transition to the state ofblock256. In theblock256 state,device19 may interrupt decoding of first asset content data and presentation of the first asset ondisplay screen41.Device19 may also identify a portion of the first asset described by metadata related to the selected graphical descriptor and locate first asset content data inmemory40 that corresponds to the identified portion.Device19 then returns to theblock250 state (as shown by path257), and may resume decoding content data using the first asset content data located instate256, and resume causing presentation of corresponding first asset content ondisplay screen41.
Ifdevice19 determines in the block253 state that the selected graphical descriptor corresponds to a second asset that is different from the first asset,device19 may transition to the state of block259 (as shown by path258). In the state ofblock259,device19 may interrupt decoding of first asset content data and presentation of the first asset ondisplay screen41.Device19 may then determine if data for the second asset is stored in memory40 (or another memory). If so, and as shown bypath260,device19 transitions to the state ofblock261. In theblock261 state,device19 may locate second asset content data inmemory40 that corresponds to the selected graphical descriptor.Device19 then returns to theblock250 states (as shown by path262), and may resume decoding content data using the second asset content data located instate261, and causes presentation of corresponding second asset content ondisplay screen41.
Ifdevice19 determines in theblock259 state that data for the second asset is not stored inmemory40, and as shown bypath263,device19 may transition to the state ofblock264. In the state ofblock264,device19 may take other appropriate action. As indicated above, such action could include prompting the user for authorization to download second asset content data over the air, to request transmission of second asset content data todevice17 for later transfer todevice19, etc.
In some embodiments, video asset data and associated graphical descriptor data may be transferred fromdevice19 todevice17 or to another device. For example,device19 may perform operations similar to those described in connection withFIG. 7. Moreover, various operations described in blocks ofFIGS. 7 and 8 might be performed elsewhere. As but one example, conversion operations described in connection with block231 (FIG. 7) could alternatively be performed indevice19 or in a third device.
The above examples only represent some embodiments of the techniques and apparatuses disclosed herein. In other embodiments, and as indicated above, other types of destination devices may operate in other types of networks. Other types of access sub-networks in which other embodiments may be implemented include (but are not limited to) FTTH passive optical networks, DSL (digital subscriber line) networks, wide area wireless networks, satellite networks, etc. Other types of destination devices include (but are not limited to) general purpose computers that may communicate with an access sub-network using one or more intermediate devices (e.g., a cable modem, a DSL modem, an Optical Network Terminal (ONT), a wide area wireless network transceiver), specialized media terminals, a media gateway, etc. Other types of transferee devices could include laptop or notebook computers, personal digital assistants (PDAs), portable media players, etc., as well as devices that might not be considered “mobile” (e.g., a desktop computer, a second STT). Transferee device could communicate withdevice17 via a direct connection to an interface ofdevice17, via a local network that includesdevice17, and/or vianetwork10 andinterface25. A destination device and one or more display devices could be combined into a single device. In at least some other embodiments utilizing different types of destination device and/or access network and/or transferee device, those devices perform operations similar to those described above in connection withdestination device17,television18 andtransferee device19.
As another example, additional embodiments include utilization of an auxiliary device to only present graphical descriptors to a user. For example, such an auxiliary device could be used to display graphical descriptors contained in one or more streams currently being received by device17 (FIG. 2). In response to seeing a graphical descriptor on the auxiliary device representing content of interest, a user might then decide to activate another device (e.g., television18) and watch that content. The auxiliary device could be dedicated to providing graphical descriptors (e.g., similar to a digital picture frame). The auxiliary device might alternatively be a device that is capable of outputting video and audio content and/or of performing other functions (e.g., a general purpose computer). In still other embodiments, a device such astelevision18 or a general purpose computer could have an operational mode in which only graphical descriptors are displayed. A user seeing a displayed descriptor representing content of interest could then place the television, computer or other device into a different mode in which the content can be viewed.
In some embodiments, a transferee device (e.g.,device19 inFIGS. 2 and 3) may not store content and descriptor data in local memory, or may only buffer such content and descriptor data as long as may be necessary for purposes of an active display. In this manner, a transferee device could be used as an additional device with which to view content and descriptors initially received bydevice17.Device17 may still store data for such content and descriptors (and/or may perform transcoding or other data conversion operations) prior to transferring same todevice19. As previously indicated,device17 may in some embodiments transfer content and descriptor data totransferee device19 without interim storage ondevice17, or may only buffer or otherwise store such data for a short period (e.g., for purposes incidental to transcoding or format conversion).
Embodiments also include one or more machine-readable storage media (e.g., a CD-ROM, CD-RW, DVD, floppy disc, FLASH memory, RAM, ROM, magnetic platters of a hard drive, etc.) that store instructions executable by one or more processors to carry out one or more of the operations described herein. As used herein (including the claims), a tangible machine-readable storage medium is a physical structure that can be touched by a human. A modulated signal would not by itself constitute a tangible machine-readable storage medium.
The foregoing description of embodiments has been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit embodiments of the present invention to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of various embodiments. The embodiments discussed herein were chosen and described in order to explain the principles and the nature of various embodiments and their practical application to enable one skilled in the art to utilize the present invention in various embodiments and with various modifications as are suited to the particular use contemplated. All embodiments need not necessarily achieve all objects or advantages identified above. Any and all permutations of various features described herein are within the scope of the invention. The features of the embodiments described herein may be combined in all possible combinations of methods, apparatus, modules, systems, and computer program products.