US20150088966A1 - Service activity user interface - Google Patents

Abstract

Described are techniques and systems for providing a user interface which provides information about one or more activities being performed by services available among affiliated devices. The information is based on activity data which may be obtained through a subscription-publication mechanism with the affiliated devices. The user interface may also provide activity controls to modify the one or more activities.

Description

PRIORITY
• This application claims priority to U.S. Provisional Patent Application Ser. No. 61/880,681 filed on Sep. 20, 2013, entitled “Enhanced Multiple Media Device Infrastructure.” The entirety of this previously filed application is hereby incorporated by reference. This matter is also known by matter numbers 579-6105 and PM6821-US.
BACKGROUND
• A media device may provide one or more services accessible to other media devices. These services may provide for various activities on the device associated with consumption of content. For example, a first media device may provide a content streaming service while a second media device provides a content display service.
BRIEF DESCRIPTION OF DRAWINGS
• FIG. 1 is a system for providing activity data about one or more activities executing on one or more devices, and presenting information about the one or more activities using an activity view user interface.
• FIG. 2 is a block diagram of activity data.
• FIG. 3 illustrates a block diagram of a media device.
• FIG. 4 illustrates a flow diagram of a process of media devices subscribing to one another to receive published activity data.
• FIG. 5 illustrates an activity view user interface.
• FIG. 6 illustrates an activity detail view interface.
• FIG. 7 illustrates an activity view interface including a private activity.
• FIG. 8 illustrates a flow diagram of a process of acquiring activity data.
• FIG. 9 illustrates a flow diagram of a process of responding to a subscription request.
• FIG. 10 illustrates a flow diagram of a process of providing an activity view user interface using the media device.
• Certain implementations and embodiments will now be described more fully below with reference to the accompanying figures, in which various aspects are shown. However, various aspects may be implemented in many different forms and should not be construed as limited to the implementations set forth herein. Like numbers refer to like elements throughout.
DETAILED DESCRIPTION
• Content such as video, music, games, applications, and so forth may be consumed using a variety of media devices. These media devices may include remote controls, televisions, set-top boxes, tablet computers, laptop computers, desktop computers, media players, home automation devices, sensors, and so forth. The media devices may be configured to communicate with one another or other devices using one or more networks. A group of media devices may be affiliated with one another. This affiliation may be based on a common account, network, geographic location, and so forth. The media devices may have one or more services which are available to provide various functions. These services may be used by the media device itself to perform one or more activities, or may be used by other media devices. For example, a set-top box may provide a streaming service which supports the activity of delivering video content to a television. The television may be running a video presentation service which supports the activity of presenting the video content on a display. Users may benefit from awareness of these activities in the affiliated devices, as well as notification of other events. For example, the user may wish to see what content the television in the next room is presenting, or what a child is reading on a tablet computer. Furthermore, the users may benefit from being able to control these activities, without necessarily opening an application dedicated to the activity. Continuing the example, the user may wish to pause the content playing on the television in the next room, without having to access a dedicated remote control application.
• Described in this disclosure are techniques and systems for exchanging activity data about one or more activities being provided by services executing at least in part on affiliated media devices. The activity data, or a portion thereof, may be provided using a subscription-publication arrangement, such that the affiliated media devices exchange properties of the various activities. In other implementations, other mechanisms such as message queues, network advertisement, broadcast, and so forth may be used to provide the activity data. The activity data, or a portion thereof, may be sent from one media device to another without request, sent upon request, or a combination thereof.
• This activity data may be used to provide an activity view interface which informs the user as to the status of the activities on the affiliated devices. The activity view interface may also provide some activity controls, allowing the user to readily modify the activities. For example, the user may transfer an activity such as presentation of content from the tablet in-hand to the television. These activity controls may be a subset of controls associated with the activity, such as pause and play for audio or video content.
• The activity view user interface may also be configured to provide additional details. For example, the user may activate a control to view additional information about the content being presented, purchase options for the content, and so forth. Controls may also be provided which invoke an application associated with the service for more comprehensive control, and so forth. For example, touching on an icon for the video content being presented on the television may open a remote control application which allows for control of the various functions on the television.
• These techniques and user interfaces described herein may improve the user experience by providing the user with concise information about the affiliated devices. This information may include content consumption activities, what devices are available to perform various activities, notifications about events, and so forth. Furthermore, the user interfaces may allow the user to make changes to the activities, such as changing which device is performing a particular activity, sharing activities or content with other devices, and so forth. The user interface is configured to emphasize the content, and support the user in consuming that content using one or more of the affiliated devices.
Illustrative System
• FIG. 1 is asystem100 for providing activity data about one or more activities and an activity view user interface based on this activity data. One or more users102(1),102(2), . . . ,102(Y) may use one or more media devices104(1),104(2), . . . ,104(D). As used in this disclosure, letters in parenthesis such as “(D)” indicate an integer value. Thesemedia devices104 may include televisions, tablet computers, personal computers, electronic book readers, gaming consoles, set-top boxes, media players, in-vehicle entertainment systems, portable media players, smartphones, servers, sensors, home automation systems, and so forth.
• A group ofmedia devices104 may be affiliated with one another. This affiliation may be based on one or more of a common account, connection to a common network, connection to a common device, geographic location, and so forth. For example, a group ofmedia devices104 may comprise thosemedia devices104 which are in a particular home, or which are associated with a single account.
• Content106 may be consumed, processed, distributed, and so forth by themedia devices104. Thecontent106 may include audio data, video data, electronic book (“eBook”) data, application programs such as games, and so forth. Thecontent106 may be downloaded or streamed from a content provider for consumption, processing, storage, and so forth on themedia device104.
• Themedia devices104 may couple to one or morelocal area networks108. Thelocal area networks108 may be personal area networks (“PANs”). The local area network(s)108 may interconnect with one another facilitating communication between themedia devices104. Thelocal area networks108 may be supported using an access point topology, ad-hoc peer-to-peer topology, and so forth. Thelocal area networks108 may be wired, wireless, or a combination. Thelocal area networks108 may be implemented using Ethernet, Wi-Fi™, Bluetooth®, ZigBee, and so forth. Within thesystem100, several differentlocal area networks108 may coexist. For example, media devices104(1) and104(6) may be connected to one another using Wi-Fi™, while the media device104(2) and104(5) may be interconnected using Bluetooth®.
• Thesystem100 may also include one or morewide area networks110. Thewide area networks110 may be communicatively coupled with one or more of theaffiliated media devices104, the local area network(s)108, and so forth. Thewide area network110 may include private networks, public networks such as the Internet, or a combination thereof. Connections may be established betweenmedia devices104 which are on the same ordifferent LANs108 orWANs110. For example, information may be passed from the media device104(2) to the media device104(1) using Bluetooth®. The media device104(1) may in turn pass this information to the media device104(7) via thewide area network110.
• Themedia devices104 may execute, or have available for execution, one ormore service modules112. Theseservice modules112 provide services for use on thelocal media device104,remote media devices104, or both. These services may include managing digital rights management (“DRM”) credentials of the content, storing and retrieving content, rendering content for presentation on an output device, and so forth. Theseservice modules112 may perform one or more activities associated with thecontent106. These activities may include receiving and processing thecontent106 for presentation, generating a stream ofcontent106 for presentation, transferring presentation from onemedia device104 to another, installing applications, accessing a uniform resource locator (“URL”), and so forth. Other services may provide for data processing, data retrieval, data storage, monitor operation of themedia devices104, such as battery status, connectivity to the networks, and so forth. In some implementations theservice modules112 may support various services such as Miracast™ as promulgated by the Wi-Fi Alliance, Universal Plug and Play (“UPnP”) as promulgated by the UPnP Forum, and so forth.
• Theservice modules112 may provideactivity data114. Theactivity data114 comprises information indicative of operation of theservice module112, such as the activities being performed. This information may include descriptions of the activity, properties of the state of the activity, and so forth. For example, theactivity data114 may indicate that a particular content title such as “The Art of Fugue” by Johann Sebastian Bach is playing on a particular media device104(2). In another example, theactivity data114 may include information indicative of an active Miracast™ session. Theactivity data114 is discussed in more detail below with regard toFIG. 2.
• Themedia devices104 may also execute anactivity manager module116. Theactivity manager module116 is configured to coordinate the acquisition and distribution ofactivity data114 from thelocal media device104 as well as fromremote media devices104. Theactivity manager module116 may be configured to operate with application programming interfaces (“API”) for theservice modules112, to exchange service commands118. The service commands118 are instructions or other information configured to affect operation of theservice module112. For example, service commands118 may include instructions to pause presentation of content, resume presentation of content, forward fast to a later point in the content, transfer the activity to anothermedia device104, and so forth.
• Theactivity manager module116 may also be configured to maintain permissions associated with the activities. These permissions determine what kind of interactionother users102,media devices104, or a combination thereof, may have with the activities. These permissions may include Read-Only, Read and Control, Private and so forth. The “Read-Only” control allows theactivity data114 associated with the activity to be distributed and provided to the otheraffiliated media devices104. In comparison, “Read and Control” allows the distribution of theactivity data114 as well as modification to the activity based on the service commands118 or other inputs. The “Private” permission prevents distribution of theactivity data114 toremote media devices104 and may also result in a denial of action based on the service commands118 or other inputs received fromremote media devices104.
• Theactivity manager module116 may be configured to operate in a peer-to-peer arrangement. In the peer-to-peer arrangement, the localactivity manager module116 may operate in conjunction with one or more remote localactivity manager modules116 to distribute theactivity data114 or other information. For example, the localactivity manager module116 may send to remoteactivity manager modules116 information following a change in the permissions for an activity. In some implementations, theactivity manager module116 may be configured to operate as a centralized authority, or operate in conjunction with another device designated as a centralized authority. For example, one of themedia devices104 may be designated as a master, and be configured to maintain information about the activities, such as the permissions, current properties of the activities, and so forth.
• Theservice module112 may register with the localactivity manager module116. The terms “local” and “remote” are relative to one another such that “local” indicates execution on a first device while “remote” indicates execution on a second device external to the first. Theservice modules112 may register one or more event handlers with theactivity manager module116, and may also provide at least a portion of theactivity data114. An activity provided by theservice module112 may also publish one or more properties using an event notification module. The event notification module provides a common service with which to publish properties such that those properties are available to other services. The event notification module is discussed below with regard toFIG. 3.
• Anactivity view module120 is configured to provide one or more user interfaces which enable theuser102 to access information about the activities on theaffiliated media devices104. Theactivity view module120 may also provide controls with which theuser102 may affect the activities on theaffiliated media devices102. Theactivity view module120 may receiveactivity data114 or information based thereon from theactivity manager module116. Upon a determination of access to the user interface provided by theactivity view module120, theactivity view module120 may requestactivity data114 about the activities from theactivity manager module116. Based on theactivity data114, a user interface such as the activityview user interface122 may be provided. The activityview user interface122 may comprise one or more of a graphical user interface, a character user interface, an audible interface, or a haptic interface. The user interfaces are discussed in more detail below with regard toFIGS. 5-7.
• Theuser102 may access theuser interface122 to view activities on the affiliated devices. For example, the user102(6) may see in the user interface information indicating that the media device104(1) is playing the movie “Life of a Cowboy”, as well as a private showing of a photo album on the local media device104(6). Likewise, the user102(1) using the media device104(1) sees in the user interface that the media device104(2) is paused while playing “The Art of Fugue”, and that the media device104(7) has someone playing the game “Whimsical Penguins”. As described, the activity controls may allow theuser102 to affect the activities, including joining those activities. For example, the user102(1) may activate the control to join the game and play “Whimsical Penguins” with the user102(7) on the media device104(7). In another example, the user102(6) may use the user interface on the media device104(6) to change permissions from “Private” to “Read-Only” for the activity of presenting the photo album, such that otheraffiliated media devices104 may present the pictures along with, or instead of, the media device104(6).
• The user interface may also provide notifications. These notifications provide information indicative of one or more services provided by theservice modules112 of theaffiliated media devices104. These notifications may include information such as battery/power status, amedia device104 re-establishing communication with the affiliated media device(s)104, and so forth. For example, the notifications may indicate the media device104(1) is running low on battery power, that communication with the media device104(7) has been established, home automation information such as status of a dishwasher, and so forth.
• Themedia devices104 may be in communication with one or more servers124(1),124(2), . . . ,124(S). Theseservers124 may provide various functions to themedia devices104. For example, theservers124 may send orstream content106, distribute updates to theservice modules112, and so forth. In some implementations theservers124 may send, receive, or send and receive one or more of theactivity data114 or the service commands118 and direct this information to one or more of the otheraffiliated media devices104. For example, theserver124 may receive activity data114(7) from the media device104(7) and provide to the media devices104(1)-(6). In another example, theserver124 may provideactivity data114 to themedia devices104.
• By using the user interface provided by theactivity view module120 and supported by theactivity manager module116, theaffiliated media devices104 may readily provide information to theusers102 about what activities are taking place, notifications for events of interest, and so forth. Theuser102 may initiate control over one or more of these activities using the interface, or may use the interface to invoke a particular application which controls that activity and operates in conjunction with theservice module112 executing the activity.
• FIG. 2 is a block diagram200 of theactivity data114 which may be acquired and managed by theactivity manager module116. As described above, theactivity data114 may be used to provide information presented by theactivity view module120 in the activityview user interface122.
• Theactivity data114 may include anactivity description202 andproperties204. Theactivity data114 may be expressed as a string, metadata, using a markup language, as executable code, script, and so forth.
• Theactivity description202 may include a service description202(1). The service description202(1) provides information indicative of theservice module112 which started the activity. For example, the service description202(1) may indicate that the service module112(1) initiated the activity. The activity identifier202(2) uniquely identifies the activity provided by theservice module112. For example, theservice module112 may be contemporaneously providing two activities such as streaming video to the media device104(1) and the media device104(6). The activity identifier202(2) provides information which disambiguates the two activities.
• Theactivity description202 may also include a description202(3). The description202(3) provides information in a human-friendly or human-readable format. For example, the description202(3) may comprise the string “Playing a movie” or “Downloading a game.”
• Permissions202(4) information may also be included in theactivity description202. The permissions202(4) provide information which indicates the permissions associated with the activity. These permissions may be used to configure how information about the activities is shared, which (if any)media devices104, user accounts, and so forth may affect the activities, and so forth. As described above, these permissions202(4) may include, but are not limited to Read-Only, Read and Control, Private, and so forth. As described above, the “Read-Only” control allows theactivity data114 associated with the activity to be distributed and provided to the otheraffiliated media devices104. In comparison, “Read and Control” allows the distribution of theactivity data114 as well as modification to the activity based on the service commands118 or other inputs. The “Private” permission prevents distribution of theactivity data114 toremote media devices104 and may also result in a denial of action based on the service commands118 or other inputs received fromremote media devices104.
• Activity type202(5) indicates whether the activity involves content delivery sharing or device controlling operation. With content delivery sharing, one media device104(9) providescontent106 to another media device104(10) for some activity, such as presentation on a display. In comparison, device controlling involves one media device104(11) providing commands which control another media device104(12). For example, the media device104(11) may send service commands118 to the media device104(12) to stop or start playback.
• Theactivity description202 may also include information on the devices involved202(6) in the activity. Continuing the prior example, in the content delivery sharing scenario, the devices involved202(6) are the media device104(9) and104(10).
• Theactivity description202 may include other information, such as version information about theservice module112 performing the activity, time the activity commenced, account(s) associated with the activity, and so forth. Theactivity description202 thus provides information about the activity, but may not provide information about other aspects of the activity, such as thecontent106 associated with the activity.
• Theproperties204 provide additional information about the activity. This information may change during the activity. For example, activity may be the media device104(1) streaming music for presentation on the media device104(5), while theproperties204 provide information about the music currently being streamed. Theproperties204 may include a content title204(1). For example, the content title204(1) may comprise the name of a song such as “The Art of Fugue”. The content description204(2) provides information about thecontent106, such as the genre of music, name of the composer, and so forth.
• Theproperties204 may provide information about the current state of the activity. Remaining time204(3) may be included in theproperties204, which indicates how much time remains in presenting thecontent106 before concluding. For example, that the song may have 1 minute 35 seconds remaining before finishing.
• Other state data204(4) may be included in theproperties204. For example, current volume setting of the output device, information about the encoding rate, and so forth.
• Portions of theactivity data114 may be sent at different times in some implementations. For example, upon start of the activity, theactivity manager module116 may distribute theactivity description202 and theproperties204 toremote media devices104. In some implementations, at a later time theproperties204 may be sent without theactivity description202. For example, theproperties204 may be distributed upon a change in one or more values, such as when the title of the song being played changes. Theproperties204 may also be distributed periodically, such as every 10 seconds. In some implementations, a combination thereof may be used.
• FIG. 3 illustrates a block diagram300 of themedia device104. Themedia device104 may include one ormore processors302 configured to execute one or more stored instructions. Theprocessors302 may comprise one or more cores. Themedia device104 may include one or more input/output (“I/O”) interface(s)304 to allow theprocessor302 or other portions of themedia device104 to communicate with other devices. The I/O interfaces304 may comprise inter-integrated circuit (“I2C”), serial peripheral interface bus (“SPI”), Universal Serial Bus (“USB”) as promulgated by the USB Implementers Forum, RS-232, one or more media device interfaces such as High Definition Multimedia Interface (“HDMI”) as promulgated by HDMI Licensing LLC, TOSLINK as promulgated by Toshiba Corp., analog video, analog audio, IEEE 3394 as promulgated by the Institute for Electrical and Electronics Engineers, and so forth.
• The I/O interface(s)304 may couple to one or more I/O devices306. The I/O devices306 may include input devices such as one or more of a camera, a microphone, a touch sensor, a button, and so forth. The I/O devices306 may also include output devices such as one or more of a display, audio speakers, haptic output device and so forth. In some embodiments, the I/O devices306 may be physically incorporated with themedia device304 or may be externally placed.
• Themedia device104 may also include one or more communication interfaces308. The communication interfaces308 are configured to provide communications between themedia device104 and other devices, such asother media devices104, routers, access points, theservers314, and so forth. The communication interfaces308 may include devices such as transceivers configured to send and receive data using the personal area networks, wired and wireless LANs, wired and wireless wide area networks (“WANs”), and so forth. For example, the communication interfaces308 may include hardware to connect using Ethernet, Wi-Fi™, Bluetooth®, ZigBee, and so forth.
• Themedia device104 may also include one or more busses or other internal communications hardware or software that allow for the transfer of data between the various modules and components of themedia device104.
• As shown inFIG. 3, themedia device104 includes one ormore memories310. Thememory310 comprises one or more computer-readable storage media (“CRSM”). The CRSM may be any one or more of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium, a mechanical computer storage medium and so forth. Thememory310 provides storage of computer readable instructions, data structures, program modules and other data for the operation of themedia device304.
• Thememory310 may include at least one operating system (OS)module312. TheOS module312 is configured to manage hardware resource devices such as the I/O interfaces304, the I/O devices306, the communication interfaces308, and provide various services to applications or modules executing on theprocessors302. Also stored in thememory310 may be one or more of the following modules. These modules may be executed as foreground applications, background tasks, daemons, and so forth.
• Acommunication module314 is configured to discoverother media devices104 and the services they support and provide a framework for establishing and using connections with theother media devices104. The connections may be authenticated, encrypted, and so forth. Thecommunication module314 may include one or more transport modules. The transport modules are configured to provide functionality at a transport layer and establish and maintain communication channels that transfer bits from onemedia device104 to another device. In one implementation the transport modules may operate atlayer 4 of the Open Systems Interconnection (“OSI”) model. Themedia device104 may have multiple transport modules available contemporaneously.
• Aregistrar module316 may be provided to work in conjunction with the other modules such as theservice modules112, theactivity manager module116, and so forth. For example, theregistrar module316 may be configured to accept information from theservice modules112 and register their presence, event handlers, and so forth on thelocal media device104.
• Anevent notification module318 may be provided to provide a mechanism for distributing theproperties204 or other information to associated devices. In some implementations theevent notification module318 may provide the functions associated with subscription, publication, or both. In other implementations, theevent notification module318 may be configured to use message queues, network advertisements, process received broadcasts, and so forth to distribute theproperties204 or other information.
• As described above, theservice modules112 provide services for use on thelocal media device104,remote media devices104, or both. Theservice modules112 may perform one or more activities associated with thecontent106. These activities may include receiving and processing thecontent106 for presentation, generating a stream ofcontent106 for presentation, transferring presentation from onemedia device104 to another, and so forth.
• A contentdirect sharing module320 may be present in thememory310. The contentdirect sharing module320 may be configured to operate in conjunction with thecommunication module314 to send a media stream to a receivingmedia device104. For example, the contentdirect sharing module320 may use connections maintained by thecommunication module314 to send audio and video data from the tablet media device104(1) to the television media device104(6) for presentation.
• Adevice controller module322 may also be present in thememory310. Thedevice controller module322 is configured to generate media presentation data. The media presentation data provides information indicative of one or more pieces ofcontent106 to be presented. For example, the media presentation data may include the uniform resource locator (“URL”) of “http://www.example.com/play?v=39A9GKd”. A first media device104(12) may generate and send the media presentation data to a second media device104(13). The second media device104(13) may then use the media presentation data to present thecontent106. Thecontent106 may be delivered as a download or stream. Thedevice controller module322 may also be configured to provide one or more service commands118. For example, thedevice controller module322 may send service commands118 to the optical disk player media device104(3) to open a disk tray.
• Thememory310 may also store theactivity manager module116 which is configured to coordinate the acquisition and distribution ofactivity data114 from thelocal media device104 as well as fromremote media devices104. Theactivity manager module116 may be configured to operate with APIs for theservice modules112, to exchange service commands118.
• Theactivity view module120 is configured to provide one or more user interfaces which enable theuser102 to access information about the activities on theaffiliated media devices102. Theactivity view module120 may also provide controls with which theuser102 may affect the activities on theaffiliated media devices102. Theactivity view module120 may receiveactivity data114 or information based thereon from theactivity manager module116. Upon a determination of access to the user interface provided by theactivity view module120, theactivity view module120 may request theactivity data114 about the activities from theactivity manager module116. Based on theactivity data114, a user interface such as the activityview user interface122 may be provided.
• Theactivity view module120 may provide the user interface to theuser102 using the I/O devices306 and may accept inputs received from the I/O devices306. The user interface may include one or more visual, audible, or haptic elements. For example, the user interface may be configured to provide a graphic user interface, an audible user interface, and so forth.
• One ormore application modules324 may be stored in thememory310. The one ormore application modules324 provide functionality which interacts with theuser102. For example, anapplication module324 may be a game which is playable by theuser102, a content player application, and so forth.
• A digital rights management module326 may provide support for presenting orprocessing content106 which is protected using one or more digital rights management schemes.Other modules328 may also be present. For example, a speech recognition module may be present and used to accept spoken input from theuser102 as received from a microphone I/O device306.
• Thememory310 may also include adatastore330 to store information. Thedatastore330 may use a flat file, database, linked list, tree, or other data structure to store the information. In some implementations, thedatastore330 or a portion of thedatastore330 may be distributed across one or more other devices including servers, network attached storage devices and so forth.
• Thedatastore330 may store thecontent106, either in its entirety or a portion.Subscription data332 may also be stored in thedatastore330. Thesubscription data332 provides information onmedia devices104 which have subscribed to publication of theactivity data114 by thelocal media device104. Thesubscription data332 may include expiration times for the subscriptions, or other information. Thedatastore330 may also store theactivity data114, service commands118, and so forth.Other data334 may also be stored, such as user preferences, configuration files, and so forth.
• FIG. 4 illustrates a flow diagram400 of a process ofmedia devices104 subscribing to one another to receive published activity data. This diagram again depicts the two devices104(1) and104(2) for illustration and not by way of limitation. Time increases down the page as indicated by thearrow402.
• Block404, at themedia device104, determines local activity provided by one or more of theservice modules112. For example, the media device104(1) determines the service module112(1) is playing a video. Likewise, the media device104(2) determines the service module112(2) is playing music.
• Block406 subscribes to otheraffiliated media devices104 to receiveactivity data114. For example, the activity manager module116(1) of the media device104(1) may initiate transmission of a subscription request408(1) from the media device104(1) to the media device104(2). Likewise, the activity manager module116(2) of the media device104(2) may initiate transmission of a subscription request408(2) from the media device104(2) to the media device104(1).
• Based on the processing of thesubscription request408, theactivity manager module116 for themedia device104 will keep those subscribedmedia devices104 informed as to the activities occurring thereon.
• In some implementations, a mechanism other than a subscription request may be used. For example, a messaging queue, broadcast, and so forth may be used to distribute theactivity data114 between the media devices.
• Block410 sendsactivity data114 to the subscribedmedia devices104. Continuing the example, the media device104(1) sends the activity data114(1) to the media device104(2). Likewise, the media device104(2) sends the activity data114(2) to the media device104(1). As described above, in some implementations a portion ofactivity data114 may be provided at a later time. For example, the current content titles204(1) being played on each of the subscribedmedia devices104 may be exchanged.
• Block412 presents an activityview user interface122. For example, theuser102 may have activated the activityview user interface122. In this example, the user102(2) of the media device104(2) takes no action; perhaps simply looking to see what other activities are taking place on theaffiliated media devices104. However, the user102(1) of the media devices104(1) decides to interact with one of the available activities as provided in theactivity data114.
• Block414 receives input to issue aservice command118 to modify the activity. For example, the user102(1) may select a control in the activityview user interface122 to request a pause in the playback of the music on the media device104(2).
• Block416 sends theservice command118 to modify the activity to theremote media device104. For example, theactivity manager116 may send the service commands118(2) to pause playback on the media device104(2).
• Based on the command, block418 modifies the activity performed by the one ormore service modules112. Continuing the example, theservice module112 playing the music on the media device104(2) may pause playback.
• Whileblocks404 through410 are depicted as occurring contemporaneously, the operations depicted need not occur symmetrically across theaffiliated media devices104. For example, media device104(2) may be configured to respond to subscription requests by publishing theactivity data114, but may not initiate subscription requests.
• FIG. 5 illustrates auser interface500 including the activityview user interface122. As described above, the activityview user interface122 may be provided by theactivity view module120. The information presented by the activityview user interface122 may be based onactivity data114 provided by theactivity manager module116. The example depicted here is from the point of view of the media device104(1).
• Theuser interface500 may include a menu502. This menu502 may include controls for performing various functions, such as executing an application, configuring themedia device104, or invoking the activityview user interface122.
• The activityview user interface122 may include one or more of anactivity view504 or anotification view506. Theactivity view504 provides information on one or more activities occurring among theaffiliated media devices104. In one implementation, theactivity view504 may be configured to present those activities involved in the consumption ofcontent106. For example, playing a movie, playing a game, presenting an eBook, and so forth. In this implementation, non-consumptive activities such as backing up data files, provisioning a device, and so forth may be omitted from theactivity view504.
• Theactivity view504 may compriseactivity tiles508. Theactivity tile508 provides a space within a graphical interface within which information about a particular activity may be presented. Theactivity tile508 may be fixed or variable in size. For example, the fixedactivity tile508 may comprise a predetermined region within the user interface. The region may be defined by a particular height and width when presented on a display device, such as arectangle500 pixels wide by 50 pixels tall. In one implementation, theactivity tile508 may be associated with a URL, such that the URL may be accessed and at least a portion of the contents retrieved therefrom may be used to populate theactivity tile508.
• Theactivity tile508 may comprise one or more standardized elements, custom elements as provided by or associated with aparticular service module112, or both. In one implementation the custom elements may be received from the one ormore service modules112. Theactivity tile508 may be configured such that information based on theactivity data114 associated with a given activity is presented within anindividual activity tile508.
• Theactivity tile508 may include anactivity graphic510. For example, the activity graphic510 may comprise an icon associated with an application, image of cover art for a music album, image of a movie poster, and so forth. The activity graphic510 may comprise one or more of a still image, animation, or video.
• Activity details512 may be presented within theactivity tile508. The activity details512 provide information about the activity. For example, the activity details512 may provide the title of thecontent106 being consumed,media devices104 involved in the activity, and so forth.
• Theactivity tile508 may also include one or more activity controls514. The activity controls514 are configured to, when activated, affect the activity represented by theactivity tile508. The activity controls514 may include controls to pause or resume playback, send to anothermedia device104 for presentation, pull from anothermedia device104 currently presenting, install an application, invoke an application associated with thecontent106, open a URL, and so forth. For example, the activity controls514 may be configured to control the activity by transferring the activity to aremote media device104, from aremote media device104, or from oneremote media device104 to anotherremote media device104. In another example, theactivity tile508 may comprise anactivity control514 configured to, upon activation byuser input516, invoke an application associated with thecontent106. The invoked application may provide a user interface with which to control consumption of thecontent106. The invocation of the application may include one or more of initiating execution, bringing the application to a foreground, transitioning user interface focus to the foreground, and so forth.
• Theactivity control514 may be standard elements provided by theactivity view module120, or custom elements. For example, theactivity control514 associated with pausing content may be represented by a standardized icon of two vertical parallel lines, or may be replaced with a customized control in aparticular activity tile508 associated with aparticular service module112.
• In some implementations the activity controls514 may present information to theuser102. For example, one of the activity controls514 configured to pause the presentation of the video playing on the media device104(1) is depicted with a pie chart depicting time remaining in thecontent106.
• The activity controls514 may provide a limited subset of controls associated with the activity. This limited subset may include the ability to pause, resume, fast forward, rewind, and so forth. For more comprehensive or detailed control over the activity, the application associated with providing a primary user interface for theservice module112 may be invoked. For example, for the activity of playing a video, a video content playback application may be accessed.
• The activity controls514 may also provide functions to send the activity to anothermedia device104, pull the activity from anothermedia device104, mirror the activity acrossmultiple media devices104, and so forth. Other functions provided may include joining a game, installing an application, photo album presentation, other activity. The activity controls514 may thus be configured to provide a simple set of adjunct controls, working with, but not necessarily replacing more comprehensive controls.
• Theuser102 may provideuser input516. Thisuser input516 may comprise one or more of activation of a key, voice command, accelerometer input, camera input, or touch input (as illustrated here). Different portions of theactivity tile508 may be configured to provide different functions. For example,user input516 associated with the activity controls514 may activate the activity controls. Other portions of theactivity tiles508 may be configured to provide other functions. For example, an activity detail control may be associated with the activity details512 which is configured to present an activity detail view, as described below inFIG. 6.
• As mentioned earlier, the activityview user interface122 may also include anotification view506. Thenotification view506 may be used to provide information which may be of use to theuser102, but which is not necessarily associated with an activity. In one implementation, thenotification view506 may present information based onactivity data114 and the corresponding one ormore properties204 for one or more services unassociated with content activities. For example, these activities may include home automation events, an alert indicating a low battery on one of themedia devices104, connectivity status of theaffiliated media devices104, and so forth.
• In some implementations theactivity view504, thenotification view506, or both may be initially presented as a single tile.User input516 may expand the single tile to depict theactivity tiles508 and so forth shown here. Similarly,user input516 may collapse theactivity tiles508 back to a single tile.
• FIG. 6 illustrates auser interface600 including an activity detail view interface. In some implementations, theuser interface600 may be invoked based onuser input516 to one or more of theactivity tiles508, such as through activation of an activity detail control. In some implementations at least a portion of theactivity tile508 may be configured as the activity detail control, such that a touch, click, and so forth, open the activity detail view depicted here. In other implementations a gesture, button input, spoken command, and so forth may invoke the activity detail view interface. The example depicted here is from the point of view of the media device104(1).
• As above, amenu602 may be provided to provide theuser102 with navigational options within the overall user interface. Theactivity detail view604 provides more comprehensive information about the particular activity. In this example, theactivity detail view604 has been invoked followinguser input516 with regard to theactivity tile508 for the activity of playing “The Art of Fugue”. As a result, theactivity detail view604 in this example provides additional information about that activity.
• Expanded activity detail606 provides additional information about the activity. In this example, this may includevarious properties204 associated with the activity, such as details about thecontent106 being presented.
• Additional information such as aprogress indicator608 depicting a relative position of the playback in thecontent106 and time remaining may be provided. The activity controls514 described above may be provided as well. In some implementations, such as on a device with a touch sensitive display, the activity controls514 may be presented in a physically larger format.
• Theactivity detail view604 may also provide additional activity controls. Adevice selection control610 provides functionality for theuser102 to change themedia devices104 involved in the activity. For example, theuser102 may select this control to pull an activity from aremote media device104 to thelocal media device104, or push the activity from thelocal media device104 to theremote media device104. Other changes may include mirroring presentation ofcontent106 tomultiple devices104, and so forth.
• Anapplication invocation control612 provides the functionality of invoking the application or other user interface associated with the activity. This application may be one configured to administer or control theservice module112 performing the activity. For example, an audio content player application may be configured to control an audiopresentation service module112. Activation of theapplication invocation control612 would thus open this application. Continuing the example illustrated here, activation of theapplication invocation control612 would launch the music player application.
• Theactivity detail view604 may also include apermission control614. Activation of thepermission control614 provides a user interface which enables theuser102 to change the permissions202(4) associated with the activity. For example, theuser102 may select to make the music presentation “Read-Only” such that the local media device104(2) may affect the activity and prevent otheraffiliated media devices104 from doing so.
• Other controls may also be presented. A purchase control may be configured to allow theuser102 to attempt to acquire rights in thecontent106 currently being presented. For example, a “buy now” button may, when activated, charge an account associated with the local media device for a purchase of thecontent106.
• The controls described in this disclosure may be activate by a touch, gesture, click, button press, spoken command, movement, and so forth. For example, the control may comprise portions of a touchscreen configured to accept input, such that when touched activation occurs.
• FIG. 7 illustrates anotheruser interface700 including an activityview user interface122 showing a private activity. The example depicted here is from the point of view of the media device104(6), and at least the media devices104(1)-(7) are affiliated with one another.
• In comparison to theuser interface500, theuser interface700 depicts five activities. One of these activities “Vacation Photo Album” has permissions202(4) set to “Private”. As described above, the “Private” permission prevents theactivity data114 associated with the activity from being provided toremote media devices104. Thus, the local media device104(1) is able to present theactivity tile508 associated with the “private” activity. The localactivity manager module116 determines, based on the permissions202(4), whatactivity data114 to provide to remote affiliatedmedia devices104. As above,user input516 may be used to invoke various functions, such as presentation of theactivity detail view604.
Illustrative Processes
• FIG. 8 illustrates a flow diagram800 of a process of acquiringactivity data114. This process may be executed at least in part by themedia device104.
• Block802 accesseslocal activity data114. As described above, thelocal activity data114 is indicative of operation of aservice module112 executing at least in part on theprocessor302 of thelocal media device104 to perform an activity. This activity may be associated withcontent106, such as presentingcontent106, streamingcontent106, receivingcontent106, and so forth.
• Block804 subscribes thelocal media device104 to one or moreremote media devices104 for delivery ofremote activity data114 from the one or moreremote media devices104. In some implementations a mechanism other than subscription may be used. For example, a message queue may be used to distribute theactivity data114. As described above, theremote activity data114 comprises information indicative of operation of a service executing at least in part on the one or moreremote media devices104 to perform an activity. The activity may be associated withcontent106.
• In some implementations, thelocal media device104 may determine one or more affiliatedremote media devices104. For example, the determination may include receiving one or more broadcasts by theremote media devices104 connected to the samelocal area network108.
• In other implementations, other mechanisms may be used to distribute theactivity data114. For example, instead of or in addition to the subscription-publication mechanism,media devices104 may periodically broadcast theiractivity data114 or a portion thereof. In another implementation, thelocal media devices104 may poll theremote media devices104 foractivity data114 or changes thereto.
• Block806 receivesremote activity data114 from the one or more subscribedremote media devices104. For example, thecommunication interface308 may receive the data by way of thelocal area network108.
• Block808 stores thelocal activity data114 and theremote activity data114. For example, this information may be stored in thememory310.
• Block810 provides at least a portion of the storedlocal activity data114, theremote activity data114, or both to one ormore service modules112 executing on theprocessor302 of thelocal media device104. For example, theactivity data114 may be provided to theactivity manager module116.
• FIG. 9 illustrates a flow diagram900 of a process of responding to a subscription request. This process may be executed at least in part by themedia device104. As described above, a subscription-publication mechanism may be used to distributeactivity data114 between theaffiliated media devices104.
• Block902 receives from one or more remote media devices104 asubscription request408. Thesubscription request408 may include information such as the network address of the requestingmedia device104. Thesubscription request408 indicates that the requestingmedia device104 would like to receive publishedactivity data114 from thelocal media device104.
• Block904, thelocal media device104 subscribes the one or more remote devices to receive theactivity data114. In some implementations the subscription may expire. For example, the subscription may expire after ten minutes. Subscribing devices would then periodically renew their subscriptions. This expiration allows foraffiliated media devices104 which have become disconnected from the network or otherwise become unavailable to be removed from the activityview user interface122.
• Block906 determines a change in thelocal activity data114. For example, one of theproperties204 may have changed, such as following a transition from playback of one song to another.
• Block908 publishes at least a portion of the changedlocal activity data114 to the subscribed one or moreremote media devices104. For example, thelocal media device104 may send updated portions of theactivity data114 using the networks to the remote affiliatedmedia devices104. This publication may include establishment of direct connections between thelocal media device104 and the subscribedremote media devices104, broadcast on the network(s), multicast on the network(s), and so forth.
• In some implementations the subscription expirations may be extended or reset following thelocal media device104 resuming operation after having been in a low power or sleep mode. This provides for themedia devices104 to enter lower power states, conserving electrical power, while maintaining their ability to participate in distribution of theactivity data114.
• As described above, activities having a permission202(4) of “Private” may not be published. As a result, they remain visible to thelocal media device104 upon which they are executing, but not to otheraffiliated media devices104.
• FIG. 10 illustrates a flow diagram1000 of a process of providing an activityview user interface122 using themedia device104. This process may be executed at least in part by themedia device104.
• Block1002 determines access to an activityview user interface122. As described above, the access may comprise presentation of the activityview user interface122 on the display device of one ormore media devices104. The access may be determined by an interrupt, handler, or other mechanism. In some implementations this may comprise receiving a request to access the activityview user interface122.
• Block1004 accessesactivity data114 for one or more of theservice modules112. Theseservice modules112 may be providing one or more activities. In one implementation, updates for at least a portion of thelocal activity data114 may be accessed. The at least a portion may comprise one ormore properties204 indicative of the activity. For example, the remaining time204(3) for video being presented.
• Block1006 provides the activityview user interface122. This interface is configured to present at least a portion of the one ormore properties204 for one or more of the activities. In some implementations, the interface may also present at least a portion of theactivity description202. The activityview user interface122 may also include one or more controls configured to control the one or more activities. For example, the activity controls514,device selection control610,application invocation control612,permission control614, and so forth As described above, the activity may be one or more of presenting video content, presenting audio content, presenting an electronic book, or executing a game.
• Block1008 may provide anotification view506 interface configured to present at least a portion of information indicative of the one or more properties of the one ormore service modules112. As described above, these notifications may be for activities unassociated withcontent106. Thus, theactivity data114 may comprise one ormore properties204 indicative of one or more services unassociated with content activities.
• Block1010 receives input indicative of activation of the one or more activity controls514. For example, theuser input516 of theuser102 touching the activity controls514 on the touchscreen.
• Block1012 sends one or more service commands118 to theservice module112 performing the activity. As described above, the one or more service commands118 are configured to instruct theservice module112 to alter performance of the activity. For example, the alteration may include pausing, resuming, rewinding, transferring the activity to anothermedia device104, and so forth.
• Those having ordinary skill in the art will readily recognize that certain steps or operations illustrated in the figures above can be eliminated or taken in an alternate order. Moreover, the methods described above may be implemented as one or more software programs for a computer system and are encoded in a computer readable storage medium as instructions executable on one or more processors.
• Embodiments may be provided as a computer program product including a nontransitory computer readable storage medium having stored thereon instructions (in compressed or uncompressed form) that may be used to program a computer (or other electronic device) to perform processes or methods described herein. The computer readable storage medium can be any one of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium and so forth. For example, the computer readable storage media may include, but is not limited to, hard drives, floppy diskettes, optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards, solid-state memory devices, or other types of physical media suitable for storing electronic instructions. Further, embodiments may also be provided as a computer program product including a transitory machine-readable signal (in compressed or uncompressed form). Examples of machine-readable signals, whether modulated using a carrier or not, include, but are not limited to, signals that a computer system or machine hosting or running a computer program can be configured to access, including signals transferred by one or more networks. For example, the transitory machine-readable signal may comprise transmission of software by the Internet.
• Separate instances of these programs can be executed on or distributed across separate computer systems. Thus, although certain steps have been described as being performed by certain devices, software programs, processes, or entities, this need not be the case and a variety of alternative implementations will be understood by those having ordinary skill in the art.
• Additionally, those having ordinary skill in the art readily recognize that the techniques described above can be utilized in a variety of devices, environments and situations. Although the present disclosure is written with respect to specific embodiments and implementations, various changes and modifications may be suggested to one skilled in the art and it is intended that the present disclosure encompass such changes and modifications that fall within the scope of the appended claims.

Claims (20)

What is claimed is:

1. A local device comprising:

a communication interface comprising a transceiver configured to send and receive data;

a display device;

a memory storing computer-executable instructions; and

a processor communicatively coupled to the communication interface, the display device, and the memory, the processor configured to execute the computer-executable instructions to:

access local activity data indicative of operation of a service executing at least in part on the processor to perform an activity associated with content, the activity comprising one or more of:

presenting video content,

presenting audio content,

presenting an electronic book, or

executing a game;

subscribe, using the communication interface, to one or more remote devices for delivery of remote activity data from the one or more remote devices, wherein the remote activity data comprises information indicative of operation of a service executing at least in part on the one or more remote devices to perform an activity associated with content;

receive, using the communication interface, remote activity data from the one or more subscribed to remote devices;

determine a request to present the user interface on the display device;

access at least a portion of the remote activity data from the one or more subscribed remote devices; and

present, with the user interface:

information about one or more of the activities, and

a control configured to change the one or more activities, responsive to input.

2. The system ofclaim 1, further comprising computer-executable instructions to:

receive input activating the control; and

send one or more service commands configured to instruct the service performing the activity to alter performance of the activity.

3. The system ofclaim 1, further comprising computer-executable instructions to:

receive from one or more remote devices a subscription request, wherein the subscription request is configured to direct the local device to publish the local activity data to the one or more remote devices;

determine a change in the local activity data; and

publish at least a portion of the changed local activity data to the subscribed one or more remote devices.

4. A computer-implemented method executable on a device, the method comprising:

receiving a request to access a user interface;

accessing activity data indicative of operation of one or more services, wherein the services are configured to perform an activity associated with content; and

presenting at least a portion of the activity data and one or more controls configured to control the activity in the user interface.

5. The computer-implemented method ofclaim 4, the user interface comprising one or more activity tiles such that each activity has an activity tile, wherein each activity tile has a predefined height and width within the user interface and the activity data for the activity is presented within the predefined height and width.

6. The computer-implemented method ofclaim 5, wherein the activity tile further comprises an activity detail control;

receiving input activating the activity detail control;

responsive to the activation of the activity detail control, providing an activity detail view comprising one or more of:

a progress indicator indicative of position of the activity within the content,

a permission control configured to accept input to set one or more permissions associated with the activity,

a device selection control, or

an application invocation control.

7. The computer-implemented method ofclaim 4, further comprising:

receiving a uniform resource locator (“URL”) from the one or more services;

retrieving one or more additional user interface elements using the URL; and

presenting the one or more additional user interface elements with the user interface.

8. The computer-implemented method ofclaim 4, wherein the one or more controls are configured to control the activity by discontinuing the activity on a local device and initiating performance of the activity on a remote device.

9. The computer-implemented method ofclaim 4, wherein the one or more controls are configured to invoke an application associated with the content.

10. The computer-implemented method ofclaim 4, the one or more services comprising applications for performing one or more of:

managing digital rights management credentials of content,

storing and retrieving content, or

rendering content for presentation on an output device.

11. The computer-implemented method ofclaim 4, wherein the activity comprises one or more of presenting or executing the content, and wherein the content comprises one or more of:

video data,

audio data,

electronic book data, or

an application program.

12. The computer-implemented method ofclaim 4, further comprising:

sending a request for activity data to one or more remote devices; and

receiving activity data from at least a portion of the one or more remote devices.

13. The computer-implemented method ofclaim 12, wherein the device and the one or more remote devices are affiliated with one another using one or more of:

an account,

geographic location,

connection to a common network, or

connection to a common device.

14. The computer-implemented method ofclaim 4, further comprising:

receiving a request for activity data from one or more remote devices; and

sending activity data to at least a portion of the one or more remote devices.

15. The computer-implemented method ofclaim 4, wherein the one or more controls include a permission control configured to accept input, the method further comprising:

receiving input using the permission control; and

setting one or more permissions for the activity, wherein the permissions are configured to affect one or more of:

publication of activity data to other devices, or

control of the activity by one or more remote devices.

16. The computer-implemented method ofclaim 4, the activity data further indicative of one or more services unassociated with content activities; and the method further comprising:

providing a notification view interface configured to present at least a portion of the activity data of the one or more services unassociated with content activities.

17. A computer-implemented method executable on a local device, the method comprising:

determining one or more affiliated remote devices;

sending a request for delivery of remote activity data to at least a portion of the one or more remote devices;

receiving the remote activity data, wherein the remote activity data comprises information indicative of operation of a service executing at least in part on the one or more remote devices to perform a remote activity;

accessing local activity data indicative of operation of a local service, wherein the local activity data comprises information indicative of operation of a service executing locally to perform a local activity;

receiving, using the communication interface, remote activity data from the one or more remote devices; and

storing activity data comprising at least a portion of the local activity data and the remote activity data.

18. The computer-implemented method ofclaim 17, wherein the local device and the one or more remote devices are communicatively coupled in a peer-to-peer configuration.

19. The computer-implemented method ofclaim 17, further comprising:

accessing the activity data; and

presenting, with the user interface, at least a portion of the activity data and one or more controls configured to control the remote activity or the local activity, wherein the user interface comprises one or more graphical user interface elements.

20. The computer-implemented method ofclaim 19, wherein the one or more controls are configured to control the activity by discontinuing the activity on a remote device and initiating performance of the activity on a local device.

Images