BACKGROUNDHome entertainment networks are becoming popular and allow media stored on or accessible by a computer functioning as a media server to be presented on more traditional presentation devices, such as a television, in a separate room from the computer. In order for a television to present such media, it is often connected with a media rendering device, which in turn is connected with the computer over a home entertainment network. In addition to providing media to a remote rendering device, the computer may supply graphical components for the rendering of a user interface (UI) on the television. The media rendering device may also generate UI components for presentation on the television. The media rendering device may composite the UI components received from the computer with the UI components generated locally on the device. The media rendering device performs this compositing operation without regard to any conflict between the UI components. Thus, the resulting presentation on the television may be composed of overlapping graphics or graphics that obscure the media presentation.
SUMMARYFunctionality is provided to a media server within a home entertainment network to intelligently coordinate the presentation of UIs generated both at the media server and locally on a networked media receiver. The media receiver may be connected with a presentation device such as a television or other display device and loudspeakers for presentation of the media and the UIs. By using a command and control backchannel over the network, the media server may receive information regarding the placement of UI components generated by the media receiver. In response, the media server may adjust the positioning of graphics, media control information, metadata, media presentation windows, and other UI components for rendering by the media receiver such that the UI components generated by the media server complement or do not interfere with the UI generated by the media receiver. Similarly, the media receiver may receive information regarding the placement of UI components generated by the media server. In response, the media receiver may adjust the positioning of UI components that are generated by the media receiver such that they complement or do not interfere with UI components generated by the media server.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Other features, details, utilities, and advantages of the claimed subject matter will be apparent from the following Detailed Description of various embodiments and implementations as further illustrated in the accompanying drawings and defined in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a schematic diagram of an implementation of a home entertainment environment.
FIG. 2 is a schematic diagram of an implementation of a communication configuration between a media server and a media receiver.
FIG. 3 is a schematic diagram of an implementation compositing on a presentation device UI and graphics from a media server and UI and graphics from a media receiver related to audio media.
FIG. 4 is a schematic diagram of an implementation compositing on a presentation device UI and graphics from a media server and UI and graphics from a media receiver related to audio/video media.
FIG. 5 is a schematic diagram of a general purpose computing system for implementing aspects of the home entertainment environment.
DETAILED DESCRIPTIONCoordination of media and UI on a presentation device may be understood in the context of a media server network.FIG. 1 depicts an exemplaryhome entertainment environment100 including aliving room102 and abedroom104. Central to thehome entertainment environment100 is amedia server106, in this implementation situated in theliving room102, but it could be located anywhere within the house. In one implementation, themedia server106 is a conventional personal computer (PC) configured to run a multimedia software package, for example, the Windows® XP Media Center Edition operating system (Microsoft Corporation, Redmond Wash.). In such a configuration, themedia server106 is able to integrate full computing functionality with a complete home entertainment system into a single PC. For example, a user can watch television (TV) in one graphical window of a video monitor, while sending email or working on a spreadsheet in another graphical window on the same monitor. In addition, themedia server106 may also include other features or components, for example: a personal video recorder (PVR) to capture live TV shows for future viewing or to record the future broadcast of a single program or series; a compact disc (CD) or digital video disc (DVD) drive108 for disc media playback; amemory drive110 for integrated storage of and access to a user's recorded content, such as TV shows, songs, pictures, and home videos; and an electronic program guide (EPG).
Instead of a conventional PC, themedia server106 may comprise a variety of other devices capable of storing and distributing media content including, for example, a notebook or portable computer, a tablet PC, a workstation, a mainframe computer, a server, an Internet appliance, or combinations thereof. Themedia server106 may also be a set-top box capable of delivering media content to a computer where it may be streamed, or the set-top box itself could stream the media content. As themedia server106 may be a full function computer running an operating system, the user may also have the option to run standard computer programs (e.g., word processing and spreadsheets), send and receive emails, browse the Internet, or perform other common functions.
In addition to storing media content, themedia server106 may be connected with a variety of media sources, for example, acable connection114, asatellite receiver116, an antenna (not shown for the sake of graphic clarity), and/or a network such as the Internet118. A user may thus control a live stream of media content (e.g., TV content) received, for example, via thecable connection114, thesatellite receiver116, or antenna. This capability is enabled by one or more tuners residing in themedia server106. The one or more tuners may alternatively be located remote from themedia server106. In either case, the user may choose a tuner to fit any particular preferences. For example, a user wishing to watch both standard definition (SD) and high definition (HD) content may employ a tuner configured for both types of contents. Alternately, the user may employ an SD tuner for SD content and an HD tuner for HD content separately.
The TV content may be received as an analog (i.e., radio frequency) signal or a digital signal (e.g., digital cable). The received TV content may include discrete content packets, where each content packet includes actual TV content (i.e., audio and video data) and a policy or policies associated with the actual TV content. If TV content is received as an analog signal, discrete content packets may be created from the analog signal.
Digital rights management (DRM) policies may be employed to protect the actual TV content or video content digitally stored on themedia server106. Licenses may therefore be associated with the actual TV or video content. A license identifies keys used to decrypt video and TV content (e.g., content packets) that are encrypted as part of DRM. In particular, the keys are used to allow consumption or use of the actual video and TV content. In certain implementations, the content packets of received TV content may be encrypted or compressed. Encrypted content packets are typically decrypted with keys transmitted to or resident at the playback device orhome network devices122,126.
Thehome environment100 may also include one or more network devices functioning asmedia receivers122,126 placed in communication with themedia server106 through anetwork128, for example, a local area network (LAN). In an exemplary embodiment, themedia receivers122,126 may be a Media Center Extender device, for example, an Xbox 360™ (Microsoft Corporation, Redmond, Wash.). Themedia receivers122,126 may also be implemented as any of a variety of conventional media rendering or computing devices, including, for example, a set-top box, a television, a video gaming console, a desktop PC, a notebook or portable computer, a workstation, a mainframe computer, an Internet appliance, a handheld PC, a cellular telephone or other wireless communications device, a personal digital assistant (PDA), or combinations thereof. Furthermore, themedia receivers122,126 may include a tuner as described above.
Thenetwork128 may comprise a wired and/or wireless network, for example, cable, Ethernet, WiFi, a wireless access point (WAP), or any other electronic coupling means, including the Internet. Thenetwork128 may enable communication between themedia server106, themedia receivers122,126, and any other connected device through packet-based communication protocols, such as transmission control protocol (TCP), Internet protocol (IP), real-time transport protocol (RTP), and real-time transport control protocol (RTCP). Communications may be transmitted directly between devices over a LAN, or they may be carried over a wide area network (WAN), for example, the Internet118.
One or more video display devices, for example amain TV120 in theliving room102, asecondary TV124 in thebedroom104, and avideo monitor112 may be situated throughout thehome environment100. These video display devices may be connected with themedia server106 via thenetwork128 either directly or via themedia receivers122,126. As shown in the example ofFIG. 1, themain TV120 and the secondary TV may be coupled to themedia receivers122,126 through conventional cables. Thevideo monitor112 may be coupled with themedia server106 directly via a video cable. Themedia server106 andmedia receivers122,126 may also or alternatively be coupled with any of a variety of video and audio presentation devices. Media content including TV content may thus be supplied to each of thevideo display devices112,120,124 over thehome network128 from themedia server106 situated in theliving room104.
Themedia receivers122,126 may be configured to receive streamed media content, including video and TV content, from themedia server106. Media content, and particularly video and TV content, may be transmitted from themedia server106 to themedia receivers122,126 as streaming media comprised of discrete content packets via any of the network protocols described above. The streamed media content may comprise video IP, SD, and HD content, including video, audio, and image files, decoded on thehome network devices122,126 for presentation on theconnected TVs120,124. The media content may further be “mixed” with additional content, for example, an EPG, presentation content related to the media content, a web browser window, and other user interface environments transmitted from the media server for output on theTVs120,124 or themonitor112. Such additional media content may be delivered in a variety of ways using different protocols, including, for example, standard remote desktop protocol (RDP), graphics device interface (GDI), or hypertext markup language (HTML).
In addition to themedia receivers122,126 and thevideo display devices112,120,124, themedia server106 may be connected with other peripheral devices, including components such as digital video recorders (DVR), cable or satellite set-top boxes, speakers, and a printer (not shown for the sake of graphic clarity). Themedia server106 may also enable multi-channel output for speakers. This may be accomplished through the use of digital interconnect outputs, such as Sony-Philips Digital Interface Format (S/PDIF) or TOSLINK® enabling the delivery of Dolby Digital, Digital Theater Sound (DTS), or Pulse Code Modulation (PCM) surround decoding.
FIG. 2 provides an overview of an implementation of aphysical configuration200 for coordinating playback of media content and presentation of user interface components, in particular audio/video content, on amedia receiver204 by themedia server202. As described above, themedia receiver204 may be connected via anetwork link208 within a local network210 (e.g., an IP network) to amedia server202. Apresentation device206, e.g., a television or monitor, may be connected with themedia receiver204 to present the media and related UI components to the user.
As indicated inFIG. 2, in general remotely sourcedmedia212 and remotely sourcedUI information214 is transmitted from themedia server202 over thenetwork link208 to themedia receiver204. Remotely sourcedmedia212 may include, for example, broadcasts of programs received and tuned by themedia server202; CD or DVD playback of music or video content by themedia server202; and music, pictures, and video stored on themedia server202. Remotely sourcedUI information212 may include, for example, menu selections, logos, and meta information about media transmitted by themedia server202, e.g., song or movie titles, album or movie art, closed captioning information, media state information (e.g., play/pause/stop indications), EPG information downloaded by themedia server202, and media duration information.
In addition to the remotely sourcedmedia212 and remotely sourcedUI information214, command andcontrol information216 may also be transmitted over thenetwork link208 between themedia server202 and themedia receiver204. This command andcontrol information216 traffic may be two-way in order to allow for coordination of a composite presentation to the user on thedisplay device206. As used herein, the term “composite” is not meant to refer to the analog video format called “composite video.” Instead, “composite” herein is meant to describe the combination or aggregation of video and UI components in a single presentation.
FIG. 3 depicts an exemplary implementation of the composition of remotely sourced audio media andremote UI information310 from amedia server302 withlocal UI326 generated at amedia receiver304 for output and display on adisplay device306 with associatedloudspeakers308. Themedia server302 and themedia receiver304 are part of ahome entertainment environment300 and are connected via anetwork316. The audio media is transmitted as an audio stream from themedia server302 to themedia receiver304 within amedia channel312 over thenetwork316. Theremote UI310 is similarly transmitted over thenetwork316 from themedia server302 to themedia receiver304 within aseparate UI channel314.
Theremote UI310 related to the audio media, for example, a music CD, is shown as generated by themedia server302. Theremote UI310 may be composed ofplayback information322 at the bottom of the screen, which in the example ofFIG. 3 includes the name of the song (Track Name), the name of the album (Album Title), play/pause indicators, the length of the song (3 min., 25 sec.), and the time elapsed in playback of the song (45 sec.). Additionally, theplayback information322 may include a graphic horizontal bar indicating the relative length of the song and a graphic vertical bar intersecting the horizontal bar and indicating the time elapsed in playback of the song. Theremote UI310 may further include a graphic orpicture324 of the album artwork associated with the particular music album.
Once the audio stream andremote UI information310 from themedia server302 are received at themedia receiver304, the audio stream is decoded by anaudio decoder module318 within the media receiver. Theaudio decoder module318 may translate the audio stream, which is generally digitally compressed, into an analog signal for transformation into sound by theloudspeakers308 associated with thedisplay device306. In alternate embodiments, theloudspeakers308 may be digital loudspeakers with the ability to directly transduce digital audio signals without intermediate decoding by theaudio decoder module318. In such embodiments, the digital audio signals may be forked by themedia receiver304 in order to provide the digital audio signals to both theloudspeakers308 and thevisualization engine320.
The analog signal may also be passed from theaudio decoder318 to avisualization engine module320 within themedia receiver304. Thevisualization engine320 may be a software program that analyzes the frequencies and intensities of the audio signals and translates the frequencies and intensities into animated graphics. An exemplarygraphic visualization326 is depicted inFIG. 3 as an animated bar graph in which the bars represent frequencies, the heights of the bars represent intensity, and the small floating bars represent a recent maximum intensity for the particular frequency. Other animated graphic visualizations for audio signals may be generated and are well known. Thegraphic visualization326 may be created by themedia receiver304 as a locally generatedUI328 for visual presentation to the user on thedisplay device306 to accompany the audio output on the associatedloudspeakers308.
Themedia receiver304 may output the remotely generatedUI310 and the locally generatedUI328 as a combinedgraphic output330 for display as acomposite UI334 on thedisplay device306. Themedia receiver304 may simply overlay the remotely generatedUI310 on top of the locally generatedUI328 or vice versa. As shown inFIG. 3, thecomposite UI334 is composed of theplayback information322 and thealbum picture324 of theremote UI310 positioned on top of thegraphic visualization326 of thelocal UI328. More complicated composition schemes are possible and additional implementations are considered with respect toFIG. 4 herein. Themedia receiver304 also provides anaudio output332 for playback on theloudspeakers308 associated with thedisplay device306.
In this exemplary implementation thegraphic visualization326 of the locally generatedUI328 is synchronized with theaudio output332 rendered by themedia receiver304. The remotely generatedUI310 is also synchronized with theaudio output332 played over theloudspeakers308 to graphically move the vertical bar of theplayback information322 horizontally and increment the time elapsed of the song while the song plays and stop both actions when the song is paused. When the song selection changes, the track name is changed in theremote UI310 component of thecomposite UI334. Further, when the album selection changes, the album title and thealbum picture324 change in theremote UI310 component of thecomposite UI334 to reflect a different album.
FIG. 4 depicts an exemplary implementation of the composition of remotely sourced audio/video media andremote UI information410 from amedia server402 withlocal UI432 generated at amedia receiver404 for output and display on apresentation device406, e.g., a video display, with associatedloudspeakers408. Themedia server402 and themedia receiver404 are part of ahome entertainment environment400 and are connected via anetwork416. The audio/video media is transmitted as a data stream from themedia server402 to themedia receiver404 within amedia channel412 over thenetwork416. Theremote UI410 is similarly transmitted over thenetwork416 from themedia server402 to themedia receiver404 within aseparate UI channel414. Additionally, themedia server402 and themedia receiver404 communicate with each other over thenetwork416 via abackchannel436 in order to coordinate the composition of the remotely generatedUI410 and the locally generatedUI432.
Theremote UI410 related to the audio/video media, for example, a movie or television program, is shown as generated by themedia server402. Theremote UI410 may be composed of avideo playback window418 at the bottom of the screen in which the movie is presented. Theremote UI410 may also includeinteractive menu graphics420 related to the software on themedia server402 for control of thehome entertainment environment400. Alogo422 identifying the presence of thehome entertainment environment400 framework may also be a part of the remotely generatedUI410. Theremote UI410 may further include a graphic or textual information (not shown) derived from metadata associated with the particular movie, for example, the movie title, the actors, director, producers, etc.
Once the audio/video stream andremote UI information410 from themedia server402 are received at themedia receiver404, the audio/video stream is decoded by an audio/video decoder module424 within the media receiver. Theaudio decoder module424 may also translate the audio/video stream, which is generally digitally compressed, into analog video signals rendered as video frames on thepresentation device406 and analog audio signals transduced into sound by the associatedloudspeakers408. In alternate embodiments, thedisplay device406 and theloudspeakers408 may be digital with the ability to directly render the digital audio/video signals without intermediate digital/analog conversion by the audio/video decoder module424, or with partial intermediate decoding by the audio/video decoder module424.
Once decoded, metadata associated with the video signal may also be passed from the audio/video decoder424 to ametadata decoder module426 within themedia receiver404. Themetadata decoder426 may be a hardware chipset or a software program that analyzes the video signal and extracts embedded data, for example,closed captioning information428. Theclosed captioning information428 or other metadata may be incorporated into a locally generatedUI432 by a graphicUI generator module430 in themedia receiver404. Themedia receiver404 may further be configured to generate additionallocal UI432 components. For example, themedia receiver404 may accept remote control or push button commands from a user to control playback of the video or volume of the audio at thepresentation device406. As depicted inFIG. 4, if a volume control command is received by themedia receiver404, thegraphic UI generator430 may generate a volume bar graphic434 to visually indicate the volume level to the user. The locally generatedUI432 may be provided for visual presentation to the user on thedisplay device406 to accompany the audio/video output from the media stream.
Themedia receiver404 may output the audio/video media, the remotely generatedUI410, and the locally generatedUI432 as a combined audio/videographic output438 for display as acomposite UI440 on thepresentation device406. The audio portion of the combined audio/videographic output438 is separated for playback on theloudspeakers408 associated with thepresentation device406.
Themedia receiver404 and the media server may coordinate the layout of the remotely generatedUI410 with the locally generatedUI428 via data exchange and command and control communications transmitted via thebackchannel436. For example, as shown inFIG. 4, thelogo422 in theremote UI410 is originally positioned at a screen location that would interfere or overlap with the volume bar graphic434 generated by thegraphic UI generator430 for thelocal UI432. In some implementations, this conflict may arise because the manufacturer of themedia receiver404 may be different from the manufacturer of the multimedia software creating thehome entertainment environment400. Themedia receiver404 may have default positions for UI components generated locally that cannot be changed. In such a situation, this information can be transmitted to themedia server402 via thebackchannel436 and theremote UI410 can be altered to accommodate thelocal UI432. This alteration can be seen in thecomposite UI440 wherein the position of thelogo422 has been adjusted downward to accommodate thevolume bar graphic434.
In the exemplary implementation ofFIG. 4, theclosed captioning information428 of the locally generatedUI432 is superimposed over the video output presented in thevideo window418 of thecomposite UI440. Thelogo422 in the remotely generatedUI410 is adjusted to accommodate the position of the volume bar graphic434 as described above. Themenu graphics420 are also presented on thecomposite UI440. Themenu graphics420 may be altered in real time to respond to user input received at themedia receiver404 and transmitted to themedia server402 via thebackchannel436. In this manner, the user can control the functions of themedia server402, for example, the selection of a new movie or song stored on themedia server402, through input at themedia receiver404. Such user input information then may dynamically affect the generation of both theremote UI410 and thelocal UI432, and thus ultimately the resultingcomposite UI440.
An exemplary hardware and operating environment for implementing the invention is depicted inFIG. 5. As indicated above, the media server and the media receiver devices may comprise general purpose computing devices. A general purpose computing device may be the form of acomputer500, including aprocessing unit502, asystem memory504, and asystem bus518 that operatively couples various system components, including thesystem memory504 to theprocessing unit502. There may be only one or there may be more than oneprocessing unit502, such that the processor ofcomputer500 comprises a single central processing unit (CPU), or a plurality of processing units, commonly referred to as a parallel processing environment. Thecomputer500 may be a conventional computer, a distributed computer, or any other type of computer; the invention is not so limited.
Thesystem bus518 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, a switched fabric, point-to-point connections, and a local bus using any of a variety of bus architectures. Thesystem memory504 may also be referred to as simply the memory, and includes read only memory (ROM)506 and random access memory (RAM)505. A basic input/output system (BIOS)508, containing the basic routines that help to transfer information between elements within thecomputer500, such as during start-up, is stored inROM506. Thecomputer500 further includes ahard disk drive530 for reading from and writing to a hard disk, not shown, amagnetic disk drive532 for reading from or writing to a removablemagnetic disk536, and anoptical disk drive534 for reading from or writing to a removableoptical disk538 such as a CD ROM, DVD, or other optical media.
Thehard disk drive530,magnetic disk drive532, andoptical disk drive534 are connected to thesystem bus518 by a harddisk drive interface520, a magneticdisk drive interface522, and an opticaldisk drive interface524, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for thecomputer500. It should be appreciated by those skilled in the art that any type of computer-readable media that can store data that is accessible by a computer, for example, magnetic cassettes, flash memory cards, digital video disks, RAMs, and ROMs, may be used in the exemplary operating environment.
A number of program modules may be stored on thehard disk530,magnetic disk532,optical disk534,ROM506, orRAM505, including anoperating system510, one ormore application programs512, for example, a multimedia software package,other program modules514, for example, a visualization engine, an audio/video decoder, a UI generator, and a metadata decoder, andprogram data516, for example, media content. A user may enter commands and information into thepersonal computer500 through input devices such as akeyboard540 andpointing device542, for example, a mouse. Other input devices (not shown) may include, for example, a microphone, a joystick, a game pad, a tablet, a touch screen device, a satellite dish, a scanner, a facsimile machine, and a video camera. These and other input devices are often connected to theprocessing unit502 through aserial port interface526 that is coupled to thesystem bus518, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB).
Amonitor544 or other type of display device is also connected to thesystem bus518 via an interface, such as avideo adapter546. In addition to themonitor544, computers typically include other peripheral output devices, such as aprinter558 and speakers (not shown). These and other output devices are often connected to theprocessing unit502 through theserial port interface526 that is coupled to thesystem bus518, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB). In the case of a media server, atuner560 may also be connected with the system bus in order to receive and convert analog content signals into digital format and similarly convert digital content into analog signals for transmission to connected devices.
Thecomputer500 may operate in a networked environment using logical connections to one or more remote computers, such asremote computer554. These logical connections may be achieved by a communication device coupled to or integral with thecomputer500; the invention is not limited to a particular type of communications device. Theremote computer554 may be another computer, a server, a router, a network personal computer, a client, a peer device, or other common network node, and typically includes many or all of the elements described above relative to thecomputer500, although only amemory storage device556 has been illustrated inFIG. 5. The logical connections depicted inFIG. 5 include a wired or wireless local-area network (LAN)550, for example, an Ethernet network, or a WiFi network, and a wide-area network (WAN)552, for example, a cable network or a telephony network. Such networking environments are commonplace in office networks, enterprise-wide computer networks, intranets and the Internet, which are all types of networks.
When used in aLAN550 environment, thecomputer500 may be connected to thelocal network550 through a network interface or adapter528, which is one type of communications device. When used in aWAN552 environment, thecomputer500 typically includes amodem548, a network adapter, or any other type of communications device for establishing communications over thewide area network552. Themodem548, which may be internal or external, is connected to thesystem bus518 via theserial port interface526. In a networked environment, program modules depicted relative to thepersonal computer500, or portions thereof, may be stored in a remote memory storage device. It is appreciated that the network connections shown are exemplary and other means of and communications devices for establishing a communications link between the computers may be used.
The technology described herein may be implemented as logical operations and/or modules in one or more systems. The logical operations may be implemented as a sequence of processor-implemented steps executing in one or more computer systems and as interconnected machine or circuit modules within one or more computer systems. Likewise, the descriptions of various component modules may be provided in terms of operations executed or effected by the modules. The resulting implementation is a matter of choice, dependent on the performance requirements of the underlying system implementing the described technology. Accordingly, the logical operations making up the embodiments of the technology described herein are referred to variously as operations, steps, objects, or modules. Furthermore, it should be understood that logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.
The above specification, examples and data provide a complete description of the structure and use of exemplary embodiments of the invention. Although various embodiments of the invention have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the spirit or scope of this invention. In particular, it should be understand that the described technology may be employed independent of a personal computer. Other embodiments are therefore contemplated. It is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative only of particular embodiments and not limiting. Changes in detail or structure may be made without departing from the basic elements of the invention as defined in the following claims.