US20130174059A1 - Communicating between a virtual area and a physical space - Google Patents

Abstract

Examples of systems and methods of communicating between a virtual area and a physical space bridge the virtual area into the physical space and bridge the physical space into the virtual area through physical apparatus located in the physical space. A virtual area may include a zone that defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone. A respective presence in the zone typically is established for each of a physical apparatus and a communicant associated with a network node. An object represents the physical apparatus in the virtual area and an avatar represents the communicant in the virtual area. The object typically is associated with an interface for interacting with the physical apparatus. At least one data stream connection typically is provisioned between the physical apparatus and the network node.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• Under 35 U.S.C. §119(e), this application claims the benefit of U.S. Provisional Application No. 61/510,698, filed Jul. 22, 2011, and U.S. Provisional Application No. 61/637,190, filed Apr. 23, 2012, the entirety of each of which is incorporated herein by reference.
• This application also relates to the following co-pending patent applications, the entirety of each of which is incorporated herein by reference: U.S. patent application Ser. No. 13/409,344, filed Mar. 1, 2012; U.S. application Ser. No. 13/229,349, filed Sep. 9, 2011; U.S. application Ser. No. 13/229,395, filed Sep. 9, 2011; U.S. application Ser. No. 13/209,812, filed Aug. 15, 2011; U.S. application Ser. No. 12/825,512, filed Jun. 29, 2010; U.S. application Ser. No. 12/694,126, filed Jan. 26, 2010; U.S. application Ser. No. 12/509,658, filed Jul. 27, 2009; U.S. application Ser. No. 12/418,243, filed Apr. 3, 2009; U.S. application Ser. No. 12/418,270, filed Apr. 3, 2009; U.S. application Ser. No. 12/354,709, filed Jan. 15, 2009; U.S. application Ser. No. 12/630,973, filed on Dec. 4, 2009; U.S. application Ser. No. 12/818,517, filed Jun. 18, 2010; U.S. patent application Ser. No. 12/855,210, filed Aug. 12, 2010; U.S. Provisional Patent Application No. 61/563,088, filed Nov. 23, 2011; and U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
BACKGROUND
• When face-to-face communications are not practical, people often rely on one or more technological solutions to meet their communications needs. These solutions typically are designed to simulate one or more aspects of face-to-face communications. Traditional telephony systems enable voice communications between callers. Instant messaging (also referred to as “chat”) communications systems enable users to communicate text messages in real time through instant message computer clients. Some instant messaging systems additionally allow users to be represented in a virtual environment by user-controllable graphical objects (referred to as “avatars”). Interactive virtual reality communication systems enable users in remote locations to communicate and interact with each other by manipulating their respective avatars in virtual spaces.
DESCRIPTION OF DRAWINGS
• FIG. 1 is a block diagram of an example of a network communications environment that includes virtual presence apparatus in a physical space, a remote client network node, and a virtual environment creator.
• FIG. 2 shows a flow diagram of a method of communicating between a virtual area and a physical space.
• FIG. 3 is a flow diagram of an example of a method performed by an example of virtual presence apparatus.
• FIG. 4 is a block diagram of an example of virtual presence apparatus.
• FIG. 5A is a block diagram of an example of virtual presence apparatus connected to a server network node.
• FIG. 5B is a block diagram of an example of virtual presence apparatus connected to a server network node.
• FIG. 5C is a block diagram of an example of virtual presence apparatus connected to a server network node.
• FIG. 6 is a flow diagram of an example of a method of administering communications between a virtual area and a physical space.
• FIG. 7 is a flow diagram of an example of a method of communicating between a virtual area and a physical space.
• FIG. 8 is a diagrammatic view of an example of a graphical interface for interfacing a user with an example of virtual presence apparatus in a physical space.
• FIG. 9 is a diagrammatic view of an example of a physical space and an example of a graphical interface for interfacing a user with an example of virtual presence apparatus in the physical space.
• FIG. 10 is a diagrammatic view of an example of a physical space and an example of a graphical interface for interfacing a user with an example of virtual presence apparatus in the physical space.
• FIG. 11 is a diagrammatic view of an example of a physical space and an example of a graphical interface for interfacing a user with an example of virtual presence apparatus in the physical space.
• FIG. 12 is a diagrammatic view of an example of a visualization of a virtual area.
• FIG. 13 is a flow diagram of an example of a method by which an example of a server network node manages communications between virtual area zones and multiple physical apparatus in respective real-world locations.
• FIG. 14 is a diagrammatic view of a network communications environment that includes network resources connected to an example of a client network node that generates an example of a graphical user interface that includes a spatial visualization of the network resources.
DETAILED DESCRIPTION
• In the following description, like reference numbers are used to identify like elements. Furthermore, the drawings are intended to illustrate major features of exemplary embodiments in a diagrammatic manner. The drawings are not intended to depict every feature of actual embodiments nor relative dimensions of the depicted elements, and are not drawn to scale.
I. Definition of Terms
• A “communicant” is a person who communicates or otherwise interacts with other persons over one or more network connections, where the communication or interaction may or may not occur in the context of a virtual area. A “user” is a communicant who is operating a particular network node that defines a particular perspective for descriptive purposes.
• A “computer” is any machine, device, or apparatus that processes data according to computer-readable instructions that are stored on a computer-readable medium either temporarily or permanently. A “computer operating system” is a software component of a computer system that manages and coordinates the performance of tasks and the sharing of computing and hardware resources. A “software application” (also referred to as software, an application, computer software, a computer application, a program, and a computer program) is a set of instructions that a computer can interpret and execute to perform one or more specific tasks. A “data file” is a block of information that durably stores data for use by a software application.
• The term “computer-readable medium” (also referred to as “memory”) refers to any tangible, non-transitory medium capable storing information (e.g., instructions and data) that is readable by a machine (e.g., a computer). Storage devices suitable for tangibly embodying such information include, but are not limited to, all forms of physical, non-transitory computer-readable memory, including, for example, semiconductor memory devices, such as random access memory (RAM), EPROM, EEPROM, and Flash memory devices, magnetic disks such as internal hard disks and removable hard disks, magneto-optical disks, DVD-ROM/RAM, and CD-ROM/RAM.
• A “window” is a visual area of a display that typically includes a user interface. A window typically displays the output of a software process and typically enables a user to input commands or data for the software process. A window that has a parent is called a “child window.” A window that has no parent, or whose parent is the desktop window, is called a “top-level window.” A “desktop” is a system-defined window that paints the background of a graphical user interface (GUI) and serves as the base for all windows displayed by all software processes.
• A “data sink” (referred to herein simply as a “sink”) is any of a device (e.g., a computer), part of a device, or software that receives data.
• A “data source” (referred to herein simply as a “source”) is any of a device (e.g., a computer), part of a device, or software that originates data.
• A “network node” (also referred to simply as a “node”) is a junction or connection point in a communications network. Examples of network nodes include, but are not limited to, a terminal, a computer, and a network switch. A “server” network node is a host computer on a network that responds to requests for information or service. A “client network node” is a computer on a network that requests information or service from a server.
• A “network connection” is a link between two communicating network nodes. A “connection handle” is a pointer or identifier (e.g., a uniform resource identifier (URI)) that can be used to establish a network connection with a network resource. A “network communication” can include any type of information (e.g., text, voice, audio, video, electronic mail message, data file, motion data stream, and data packet) that is transmitted or otherwise conveyed from one network node to another network node over a network connection.
• Synchronous conferencing refers to communications in which communicants participate at the same time. Synchronous conferencing encompasses all types of networked collaboration technologies, including instant messaging (e.g., text chat), audio conferencing, video conferencing, application sharing, and file sharing technologies.
• A “communicant interaction” is any type of direct or indirect action or influence between a communicant and another network entity, which may include for example another communicant, a virtual area, or a network service. Examples of types of communicant communications include communicants communicating with each other in realtime, a communicant entering a virtual area, and a communicant requesting access to a resource from a network service.
• “Presence” refers to the ability and willingness of a networked entity (e.g., a communicant, service, or device) to communicate, where such willingness affects the ability to detect and obtain information about the state of the entity on a network and the ability to connect to the entity.
• A “realtime data stream” is data that is structured and processed in a continuous flow and designed to be received with no delay or only imperceptible delay. Realtime data streams include digital representations of voice, video, user movements, facial expressions and other physical phenomena, as well as data within the computing environment that may benefit from rapid transmission, rapid execution, or both rapid transmission and rapid execution, including for example, avatar movement instructions, text chat, realtime data feeds (e.g., sensor data, machine control instructions, transaction streams and stock quote information feeds), screen shares, and file transfers.
• A “physical space” is a three-dimensional real-world environment in which a communicant can be located physically.
• A “virtual area” (also referred to as an “area” or a “place”) is a representation of a computer-managed space or scene. Virtual areas typically are one-dimensional, two-dimensional, or three-dimensional representations; although in some examples a virtual area may correspond to a single point. Oftentimes, a virtual area is designed to simulate a physical, real-world space. For example, using a traditional computer monitor, a virtual area may be visualized as a two-dimensional graphic of a three-dimensional computer-generated space. However, virtual areas do not require an associated visualization. A virtual area typically refers to an instance of a virtual area schema, where the schema defines the structure and contents of a virtual area in terms of variables and the instance defines the structure and contents of a virtual area in terms of values that have been resolved from a particular context.
• A “persistent virtual area” is a virtual area that persists even after all communicants have disconnected from the virtual area. The state of a persistent virtual area is preserved so that it can be restored the next time a communicant connects to the virtual area. A “persistent association” between a virtual area and virtual presence apparatus is an association that persists even after all communicants and the virtual presence apparatus have disconnected from the virtual area.
• A “virtual area application” (also referred to as a “virtual area specification”) is a description of a virtual area that is used in creating a virtual environment. A virtual area application typically includes definitions of geometry, physics, and realtime switching rules that are associated with one or more zones of the virtual area.
• A “virtual area enabled communications application” is a client communications application that integrates realtime communications (e.g., synchronous conferencing functionalities, such as audio, video, chat, and realtime other data communications) with a virtual area.
• A “virtual environment” is a representation of a computer-managed space that includes at least one virtual area and supports realtime communications between communicants.
• A “position” in a virtual area refers to a location of a point or an area or a volume in the virtual area. A point typically is represented by a single set of one-dimensional, two-dimensional, or three-dimensional coordinates (e.g., x, y, z) that define a spot in the virtual area. An area typically is represented by the three-dimensional coordinates of three or more coplanar vertices that define a boundary of a closed two-dimensional shape in the virtual area. A volume typically is represented by the three-dimensional coordinates of four or more non-coplanar vertices that define a closed boundary of a three-dimensional shape in the virtual area.
• VoIP (Voice over Internet Protocol) refers to systems and methods of delivering voice and other communications over Internet Protocol (IP) networks.
• As used herein, the term “includes” means includes but not limited to, the term “including” means including but not limited to. The term “based on” means based at least in part on.
II. Communicating Between a Virtual Area and a Physical Space
• Examples that are described herein provide systems and methods of communicating between a virtual area and a physical space. These examples bridge the virtual area into the physical space and bridge the physical space into the virtual area through virtual presence apparatus (VPA) located in the physical space. Examples of the virtual presence apparatus transduce human perceptible stimulus (e.g., audio, visual, mechanical, and other sensory stimulus) between the virtual area and the physical space such that communicant interactions in the virtual area are seamlessly integrated into the physical space and vice versa.
• FIG. 1 shows an embodiment of an exemplarynetwork communications environment10 that includes avirtual presence apparatus12 in aphysical space14, a remoteclient network node16, and avirtual environment creator18 that are interconnected by a network (not shown) that supports the transmission of a wide variety of different media types (e.g., text, voice, audio, video, and other data) between network nodes. The network connections between network nodes may be arranged in a variety of different stream handling topologies, including a peer-to-peer architecture, a server-mediated architecture, and hybrid architectures that combine aspects of peer-to-peer and server-mediated architectures. Exemplary topologies of these types are described in U.S. Pat. Nos. 7,769,806 and 7,844,724.
• Theclient network node16 includes input/output (I/O) hardware, a processor, and a computer-readable memory that stores aninstance20 of at least one virtual area enabled communications application that is executable by the processor. Thecommunications application20 typically provides graphical interface and communications functions for communicating with thevirtual presence apparatus12, thevirtual environment creator18, and other client network nodes in connection with one or more virtual areas. Examples of the communications applications are described in U.S. application Ser. No. 12/418,243, filed Apr. 3, 2009, U.S. application Ser. No. 12/630,973, filed Dec. 4, 2009, U.S. application Ser. No. 12/354,709, filed Jan. 15, 2009, U.S. application Ser. No. 12/509,658, filed Jul. 27, 2009, U.S. application Ser. No. 13/209,812, filed Aug. 15, 2011, and U.S. application Ser. No. 13/229,349, filed Sep. 9, 2011. Theclient network node16 has a respective set of one or more sources and a respective set of one or more sinks. Exemplary sources include an audio source (e.g., an audio capture device, such as a microphone), a video source (e.g., a video capture device, such as a video camera), a chat source (e.g., a text capture device, such as a keyboard), a motion data source (e.g., a pointing device, such as a computer mouse), and other sources (e.g., file sharing source or a source of a customized real-time data stream). Exemplary sinks include an audio sink (e.g., an audio rendering device, such as a speaker or headphones), a video sink (e.g., a video rendering device, such as a display monitor), a chat sink (e.g., a text rendering device, such as a display monitor), a motion data sink (e.g., a movement rendering device, such as a display monitor), and other sinks (e.g., a printer for printing shared files, a device for rendering real-time data streams different from those already described, or software that processes real-time streams for analysis or customized display). Theclient network node16 also typically includes administrative policies, user preferences (including preferences regarding the exportation of the user's presence and the connection of the user to thevirtual environment creator18 and other communicants), and other settings that define a local configuration that influences the administration of realtime connections with thevirtual presence apparatus12, thevirtual environment creator18, and other network nodes.
• Thevirtual presence apparatus12 is located in thephysical space14. In the illustrated example, thevirtual presence apparatus12 is positioned on a table22 in a real-world conference room containing fivecommunicants24,26,28,30,32. The four communicants24-30 are seated around the table22, and thefifth communicant32 is standing beside a real-world view screen34. Three of the seated communicants24-28 are operating respective virtual area enabled communications applications on theirclient network nodes36,38,40 (e.g., mobile computers, such as laptop computers, tablet computers, and mobile phones) through which they are connected to the remoteclient network node16; these three communicants24-28 and the other twocommunicants30,32 are connected to the remoteclient network node16 through thevirtual presence apparatus12.
• Thevirtual presence apparatus12 typically includes software and hardware resources that enable thevirtual presence apparatus12 to connect to thevirtual environment creator18 and the remoteclient network node16, either directly (e.g., peer-to-peer) or through a hosted network connection. In some examples, thevirtual presence apparatus12 or a network node hosting the virtual presence apparatus includes a complete or modified version of thecommunications application20, which provides functions for communicating with thevirtual environment creator18 and establishing network connections and communicating realtime data streams with the client network nodes. When connected to thevirtual environment creator18, thevirtual presence apparatus12 can be registered in association with and/or logged into the one or more virtual areas. When logged into a virtual area, thevirtual presence apparatus12 transduces human perceptible stimulus (e.g., audio, visual, mechanical, and other sensory stimulus) between the client network nodes of communicants who are present in the virtual area and thephysical space14. In this way, thevirtual presence apparatus12 bridges a physical experience of thephysical space14 to communicants in the one or more virtual areas (i.e., communicants who are present in the virtual areas) and bridges communicant interactions in the one or more virtual areas to communicants in thephysical space14.
• In the illustrated example, thevirtual environment creator18 includes at least oneserver network node42 that provides a networkinfrastructure service environment44 that manages sessions of the remoteclient network node16 and thevirtual presence apparatus12 in one or morevirtual areas46 in accordance with respectivevirtual area applications48. Each of thevirtual area applications48 is hosted by a respective one of thevirtual areas46 and includes a description of the respectivevirtual area46. Communicants operating respective client network nodes connect to thevirtual area applications48 through virtual area enabled communications applications.
• A virtual area typically includes one or more zones. A zone may be a rendered spatial extent, a set of rules applied to a spatial extent, or both. Zones may be arranged hierarchically in a virtual area, with an outermost zone (referred to herein as the “Global Governance Zone”) enclosing all other zones in the virtual area. Within the Global Governance Zone, there can be location zones (e.g., rooms of a virtual area) or smaller governance zones that enclose a group of location zones and provide regions of governance on the map. A zone definition typically also includes one or more channel definitions that describe how to create respective channels in the zone and specify the information about the channel that is published to a client network node that becomes present in the zone. A channel is always uniquely defined point-to-point and is unique to a virtual area application and a session between a client network node and the virtual area platform.
• Examples of the types of rules that may be associated with a zone include switching rules, governance rules, and permission rules.
• Switching rules govern realtime stream connections between network nodes that are linked to the virtual area (e.g., network nodes that are associated with objects, such as avatars, in the virtual area). The switching rules typically include a description of conditions for connecting sources and sinks of realtime data streams in terms of positions in the virtual area. Each switching rule typically includes attributes that define the realtime data stream type to which the rule applies and the location or locations in the virtual area where the rule applies. In some examples, each of the rules optionally may include one or more attributes that specify a required role of the source, a required role of the sink, a priority level of the stream, and a requested data routing topology. In some examples, if there are no explicit switching rules defined for a particular part of the virtual area, one or more implicit or default switching rules may apply to that part of the virtual area.
• Governance rules control who has access to resources (e.g., the virtual area itself, regions with the virtual area, and objects within the virtual area), who has access to data (e.g., data streams and other content) that is associated with the virtual area, what is the scope of that access to the data associated the virtual area (e.g., what can a user do with the data), and what are the follow-on consequences of accessing that data (e.g., record keeping, such as audit logs, and payment requirements). In some examples, an entire virtual area or a zone of the virtual area is associated with a “governance mesh” that enables a software application developer to associate governance rules with a virtual area or a zone of a virtual area. This avoids the need for the creation of individual permissions for every file in a virtual area and avoids the need to deal with the complexity that potentially could arise when there is a need to treat the same document differently depending on the context.
• A permission rule defines a respective capability requirement (e.g., for a respective action, behavior, or state) in terms of one or more capabilities, attributes, and settings, which may be persistent or transient. Examples of capabilities systems for administering permission rules are described in U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
• In some examples, a virtual area is defined by a specification that includes a description of geometric elements of the virtual area and one or more rules, including switching rules and governance rules. Examples of virtual area specifications are described in U.S. application Ser. No. 12/418,243, filed Apr. 3, 2009, U.S. application Ser. No. 12/818,517, filed Jun. 18, 2010, U.S. patent application Ser. No. 12/855,210, filed Aug. 12, 2010, and U.S. Provisional Application No. 61/563,088, filed Nov. 23, 2011.
• The networkinfrastructure service environment44 typically includes one or more network infrastructure services that cooperate with the virtual area enabledcommunications application20 to establish and administer network connections between thevirtual presence apparatus12, the remoteclient network node16, and other network nodes. Among the network infrastructure services that are included in an exemplary embodiment of the networkinfrastructure service environment44 are an account service, a security service, an area service, a rendezvous service, and an interaction service. The structure, operation, and components of such an embodiment of the networkinfrastructure service environment44 are described in U.S. patent application Ser. No. 12/825,512, filed Jun. 29, 2010.
• The virtual area enabledcommunications applications20, thearea applications48, and the networkinfrastructure service environment44 together provide a platform (referred to herein as “the platform”) that administers the realtime connections with network nodes in a virtual area subject to a set of constraints that control access to the virtual area instance.
• The platform tracks communicants' realtime availabilities and activities across the different communication contexts that are defined by thearea applications48. This information is presented by the virtual area enabled communications applications to the communicants in the form of realtime visualizations that enable the communicants to make more informed network interaction decisions (e.g., when to interact with a contact) and encourages the communicants to initiate interactions with other communicants and to join contexts (e.g., an ongoing conversation between communicants) of which the communicants otherwise would not have been aware. In some embodiments, the realtime visualization includes visual cues as to the presence and activities of the communicants in the contexts of thearea applications48. The presentation of these visual cues typically depends on one or more of governance rules associated with thevirtual areas46, administrative policies, and user preferences (including preferences regarding the exportation of the user's presence and the connection of the user to areas and other communicants), which may define tiered relationship based predicates that control access to presence information and/or network resources on a zone-by-zone basis.
• In some embodiments, theserver network node42 remotely manages client communication sessions with each other and with thevirtual presence apparatus12, and remotely configures audio and graphic rendering engines on the client network nodes, as well as switching of data streams by sending instructions (also referred to as definitions) from the remotely hostedarea applications48 to the client network nodes in accordance with the stream transport protocol described in U.S. application Ser. No. 12/825,512, filed Jun. 29, 2010. Data is shared between theclient network node16 and other network nodes as definition records over transport protocol sockets. Theclient communications application16 receives configuration instructions from theserver node42 through definition records that are received over a server session between theclient network node16 and theserver network node42. In some of these examples, theserver network node42 sends to each of the client network nodes provisioning messages that configure the client network nodes to interconnect respective data streams between active ones of their complementary sources and sinks over respective peer-to-peer (P2P) sessions in accordance with switching rules specified in thearea applications48 and the locations where the communicants and the virtual presence apparatus are present in thevirtual area46. Theclient network node16 sends content to and receives content from other network nodes through definition records that are transmitted on content-specific channels on respective sessions with the other network nodes. Data is shared in accordance with a publish/subscribe model. A stream transport service on theclient network node16 subscribes only to the data that are needed by the client network node. To subscribe, the stream transport service negotiates a channel on a session that is established with another network node. The channel is negotiated by well-known GUID for theparticular area application48. Definition records are transmitted only when a subscriber exists on the other end of a transport protocol socket. Definition records that are received by the stream transport service are delivered to the subscribing ones of the client communications application processes on arrival. In this way, theserver network node42 connects thevirtual presence apparatus12 to thevirtual area46 so that thevirtual presence apparatus12 can bridge a physical experience of thephysical space14 to communicants in thevirtual area46 and bridge a physical experience of communicant interactions in thevirtual area46 to communicants in thephysical space14.
• In the illustrated embodiment, thecommunications application20 operating on the remoteclient network node16 presents a respectivespatial visualization50 of thevirtual area46 in accordance with data received from the networkinfrastructure service environment44. Thecommunications application20 also provides a graphical interface for receiving user commands and providing a spatial interface that enhances the realtime communications between the communicants. Thespatial visualization50 includes respectivegraphical representations52,54,56,58 (referred to herein as “avatars” or “sprites”) of the communicants who are present in thevirtual area46 in spatial relation to agraphical representation59 of thevirtual area46. In the illustrated example, thesprites52,54,56 represent the threecommunicants24,26,28 (Beth, Fran, Art) who are seated in thephysical space14 and are operating the localclient network nodes36,38,40, and thesprite58 represents the communicant (Ed) who is operating the remoteclient network node16. Thespatial visualization50 may include other objects (also referred to as “props”). Examples of such objects include aview screen object60 for interfacing with application sharing functions of the communications application20 (as described in, e.g., U.S. application Ser. No. 12/418,270, filed Apr. 3, 2009), atable object62 for interfacing with file sharing functions of thecommunications application20, and aVPA object64 for interfacing with thevirtual presence apparatus12 in thephysical space14. Thespatial visualization50 typically is presented in arespective window66 that is generated by thecommunications application20 on a “desktop” or other system-defined base window on thedisplay hardware68 of the remoteclient network node16.
• The activities of the communicants in thevirtual area46 may be inferred from the activities on the various communication channels over which the respective client network nodes are configured to communicate. The activities on the communication channels are represented in the graphical interface by visual cues that are depicted in association with the graphical representations52-58 of the communicants. For example, the “on” or “off” state of a communicant's local speaker channel is depicted by the presence or absence of a headphones graphic61 on the communicant's sprite. When the speakers of the communicant who is represented by the sprite are on, the headphones graphic61 is present (see sprites Beth and Fran) and, when the communicant's speakers are off, the headphones graphic61 is absent. The “on” or “off” state of the communicant's microphone is depicted by the presence or absence of a microphone graphic63 on the communicant's sprite. When the microphone is on, the microphone graphic63 is present (see sprite Fran); and, when the microphone is off, the microphone graphic63 is absent. The headphones graphic61 and the microphone graphic63 provide visual cues of the activity states of the communicant's sound playback and microphone devices. The “on” or “off” state of the communicant's microphone is depicted by the presence or absence of a microphone graphic on the communicant's graphic representation and a series ofconcentric circles65 that dynamically radiate away from the communicant's graphic representation in a series of expanding waves. When the microphone is on, the microphone graphic63 and the radiatingconcentric circles65 are present and, when the microphone is off, the microphone graphic63 and the radiatingconcentric circles65 are absent. In addition to or alternatively, the current activity on a communicant's microphone channel is indicated by a dynamic visualization that lightens and darkens the communicant's avatar in realtime to reflect the presence or absence of audio data on the microphone channel. Thus, whether or not their local speakers are turned on, communicants can determine when another communicant is speaking by the “blinking” of the coloration of that communicant's avatar.
• The activity on a communicant's text chat channel is depicted by the presence or absence of the hand graphic67 adjacent the communicant's sprite (see sprite Ed). Thus, when a communicant is transmitting text chat data to another network node the hand graphic67 is present, and when a communicant is not transmitting text chat data the hand graphic67 is not present. In some embodiments, text chat data is transmitted only when keyboard keys are depressed, in which case the visualization of the communicant's text channel appears as a flashing on and off of the hand graphic67.
• Theview screen prop60 is associated with application sharing functionality of the platform that enables communicants to share applications operating their respective client network nodes. The application sharing functionality is invoked by activating a view screen (e.g., by single-clicking the view screen object with an input device). In some embodiments, the platform provides visual cues that indicate whether or not a communicant is sharing an application over an application sharing channel. In response to a communicant's selection of the view screen prop, the communicant's sprite automatically is moved to a position in the graphical representation of the virtual area that is adjacent the view screen prop. The position of a communicant's sprite adjacent the view screen prop indicates that the communicant currently is sharing or is about to share an application with the other communicants in the virtual area. Other communicants in the virtual subscribe to the shared application data by selecting the view screen prop in their respective views of thespatial visualization50. The avatar of each communicant who is viewing a shared application is depicted with a pair of “eyes” to indicate that the represented communicants are viewing the content being shared in connection with the view screen props. The graphical depiction of view screen prop is changed depending on whether or not an active application sharing session is occurring. For example, the depicted color of the view screen may change from a brighter color during an active application sharing session to a darker color when there is no application sharing taking place. Examples of the application sharing process are described in connection with FIGS. 26-28 of U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, and in U.S. patent application Ser. No. 12/418,270, filed Apr. 3, 2009.
• FIG. 2 shows an example of a method by which thevirtual environment creator18 and thevirtual presence apparatus12 bridge the virtual area into the physical space and bridge the physical space into the virtual area. In accordance with this method, thevirtual presence apparatus12 transforms human perceptible physical stimuli in a physical space into physical space data streams of different respective data stream types (FIG. 2, block101). Theserver network node42 publishes respective ones of the physical space data streams in a zone of a virtual area in a virtual communications environment (FIG. 2, block103). The zone defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone. Theserver network node42 establishes a respective presence in the zone for each of multiple communicants associated with respective client network nodes. Each of one or more of the client network nodes publishes one or more respective client data streams (FIG. 2, block105). Theserver network node42 provisions data stream connections for transmitting respective ones of the published physical space data streams to respective ones of the client network nodes, transmitting respective ones of the published client data streams to respective ones of the client network nodes, and transmitting respective ones of the published client data streams to the physical space (FIG. 2, block107). Thevirtual presence apparatus12 transforms the published client data streams transmitted to the physical space into human perceptible physical stimuli in the physical space (FIG. 5, block109).
• FIG. 3 shows an example of a process that is implemented by an example of thevirtual presence apparatus12. In accordance with this method, thevirtual presence apparatus12 transmits a globally unique identifier of thevirtual presence apparatus12 for association with a virtual area by a network service administering the virtual area (FIG. 3, block90). Thevirtual presence apparatus12 generates output data from human perceptible stimulus in a physical space (FIG. 3, block92), and transmits the output data in connection with the virtual area (FIG. 3, block94). Thevirtual presence apparatus12 receives input data associated with the virtual area (FIG. 3, block96), and generates human perceptible stimulus in the physical space from the input data (FIG. 3, block98).
• FIG. 4 shows an example70 of thevirtual presence apparatus12 that includes aninput transducer72, anoutput transducer74, acommunication interface76, a computer-readable memory78 that stores a globally unique identifier of thevirtual presence apparatus70, and aprocessor80. Thecommunication interface76 transmits anoutput signal86 and receives aninput signal88.
• Thevirtual presence apparatus70 may be implemented in a variety of different ways. In some examples, thevirtual presence apparatus70 is composed of multiple components (e.g., two or more of a speaker, a microphone, a light projector, and a camera) that are integrated into a unitary device. In other examples, thevirtual presence apparatus70 is composed of a central hub (e.g., a virtual area enabled network switch or router) that controls and configures one or more separate and distinct peripheral components (e.g., a speakerphone, a digital projector, a camera, and a remote-controlled laser pointer) that are connected to respective ports (e.g., Universal Serial Bus (USB) ports) of the hub. Examples of thevirtual presence apparatus70 may have different industrial designs. In some examples, thevirtual presence apparatus70 has the form factor of a desktop appliance (e.g., a form factor similar to that of a computer, speakerphone, a digital projector, or a network hub), whereas other examples of thevirtual presence apparatus70 have robotic form factors (e.g., a remote-controlled electro-mechanical machine, which may or may not have a humanoid appearance).
• Theinput transducer72 generates output data from humanperceptible stimulus82 in thephysical space14. Theinput transducer72 typically generates the output data from human perceptible stimulus that is broadcasted into the physical space. Depending on the desired communication application, theinput transducer72 may generate output data from one or more human perceptible stimuli, including for example audio, visual, mechanical, and other sensory stimuli. In some examples, theinput transducer72 includes one or more of an acoustic-to-electric transducer (e.g., a microphone, which may be a component of a telephony device, such as a mobile phone or a VoIP phone, or a headset), a light-to-electric transducer (e.g., a camera, such as a still image camera, a video camera, and a scanner that scans physical documents into scanned images), an electric-to-electric transducer (e.g., a touchscreen or other touch-sensitive sensor equipped with resistive, capacitive, surface acoustic wave, optical, or other touch-sensitive technologies), a mechanical-to-electric transducer (e.g., a tactile or other pressure- or force-sensitive transducer, a texture-sensitive transducer), and a chemical-to-electric transducer (e.g., a olfactory sensor that is capable of detecting one or more odorants).
• Theoutput transducer74 generates humanperceptible stimulus84 in thephysical space14. Theoutput transducer74 typically broadcasts the human perceptible stimulus into the physical space. Depending on the desired communications application, theoutput transducer74 may generate one or more human perceptible stimuli from input data, including for example audio, visual, mechanical, and other sensory stimuli. In some examples, theoutput transducer74 includes one or more of an electric-to-acoustic transducer (e.g., a speaker, which may be a component of a telephony device, such as a mobile phone or a VoIP phone, or a headset), an electric-to-light transducer (e.g., an image projector such as a digital projector, a touchscreen display, a light beam projector such as a laser pointer, or a three-dimensional hologram generator), an electric-to-mechanical transducer (e.g., a haptic transducer, an electric motor that moves mechanical components, such as light sources and robot tools, and other components in the physical space, and a printer that outputs printed documents or three-dimensional objects), and an electric-to-chemical transducer (e.g., an electric odorant delivery system).
• Thevirtual presence apparatus70 bridges communicant activity in thephysical space14 into thevirtual area46 and bridges communicant activity in the virtual area into thephysical space14. In this process, thevirtual presence apparatus70 typically encodes output data generated by theinput transducer72 from communicant activity in thephysical space14 into theoutput signal86 that is sent to theremote network node16 connected to the virtual area, and decodes theinput signal88, which is received from theremote network node16 and relates to communicant activity in the virtual area, into input data that is sent to theoutput transducer74.
• Thevirtual presence apparatus12 typically is registered with theserver network node42 before thevirtual presence apparatus12 can be logged into a server session with theserver network node42. In some examples, thevirtual presence apparatus12 includes hardware and software resources that enable it to register directly with theserver network node42.
• For example,FIG. 5A shows an example of a network connection between theserver network node42 and an example85 of thevirtual presence apparatus12 that can register directly with theserver network node42.
• In other examples, a host computer (e.g., one of the client network nodes36-40 in the physical space) registers thevirtual presence apparatus12 with theserver network node42.FIG. 5B shows an example of a network connection between theserver network node42 and an example87 of thevirtual presence apparatus12 that is hosted by aclient network node89 in thephysical space14. In this example, theclient network node89 submits to theserver network node42 registration information and login requests on behalf of both thevirtual presence apparatus87 and the communicant who uses theclient network node89 to access thevirtual area46.
• FIG. 5C shows an example of a network connection between theserver network node42 and an example of93 of thevirtual presence apparatus12 that includes one or more integral components of aclient network node91 in thephysical space14. Thevirtual presence apparatus93 typically includes one or more hardware and software resources of theclient network node91. In some examples, thevirtual presence apparatus93 includes software that resides in the memory of theclient network node91 and is executed by the processor of theclient network node91 to leverage hardware resources of theclient network node91 in the process of integrating communicant interactions in the virtual area into the physical space. In some of these examples, hardware resources of theclient network node91 are partitioned (e.g., by a hypervisor or virtual machine monitor that reserves a respective set of client system resources for each partition or virtual machine) into a set of hardware resources that are used by the virtual area enabledcommunications application20 and a separate set of hardware resources that constitute elements of thevirtual presence apparatus93. For example, a peripheral headset may be reserved for use by the virtual area enabledcommunications application20, whereas separate microphone and speaker hardware may be reserved for use by thevirtual presence apparatus93. In some examples, certain hardware resources of the client network node91 (e.g., a camera, a hard drive memory, or an optical disk drive) that are allocated to thevirtual presence apparatus93 are associated with respective objects in thevirtual area46, allowing those resources to be shared by other communicants in thevirtual area46.
• During registration, thevirtual presence apparatus12 transmits (either directly, or indirectly through a network node hosting the virtual presence apparatus12) registration data through its communication interface to theserver network node42. The registration data typically includes the globally unique identifier of thevirtual presence apparatus12 and configuration data. The configuration data may include, for example, a device type identifier, an indication whether thevirtual presence apparatus12 should be associated with an existing virtual area or a new virtual area, one or more conditions on the availability of the associated virtual area (e.g., the associated virtual area is accessible to communicants conditioned on thevirtual area apparatus12 being present in or logged into the virtual area), a specification of the source and sink capabilities of thevirtual presence apparatus12, and a specification of a graphical representation of thevirtual presence apparatus12. Based on the registration data, theserver network node42 generates one or more database records that store the registration information, including the identifier of thevirtual presence apparatus12 and an identifier of the new or existing virtual area. The one or more database records create a persistent association between thevirtual presence apparatus12 and the virtual area. The virtual presence apparatus identifier typically is registered with theserver network node42 independently of any communicant identifier. Theserver network node42 determines the source and sink capabilities of the virtual presence apparatus, either directly (e.g., from the configuration data) or indirectly (e.g., by using the device type identifier for thevirtual presence apparatus12 as an index into a device capabilities table).
• In some examples, thevirtual presence apparatus12 is associated with a virtual area independently of any particular communicant such that it is available as a resource for any communicant who is present in the virtual area. In this way, the virtual presence apparatus functions as a prop or a fixture of the associated virtual area, which is tied to the physical location of the virtual presence apparatus. In some examples, the association between the virtual presence apparatus and the virtual area is such that the virtual area is inaccessible until after the virtual presence apparatus has been logged into the networkinfrastructure service environment44. In some of these examples, communicants cannot establish a presence in the associated virtual area (and, in some cases, may not even be presented with an option for entering the virtual area such that the virtual area does not appear to exist) until after the virtual presence apparatus has been connected to the virtual area by the networkinfrastructure service environment44. These examples allow communicants to establish a persistent association between a virtual area and a particular physical space by leaving the virtual presence apparatus in the same physical space, thereby leveraging the persistent spatial association with the real-world location of the physical space to further strengthen the bridging between the virtual area and the physical space.
• After thevirtual presence apparatus12 has been registered with theserver network node42, thevirtual presence apparatus12 can be logged into the networkinfrastructure service environment44. Thevirtual presence apparatus12 can either log itself into the networkinfrastructure service environment44 automatically each time it is turned on or it can be logged into the networkinfrastructure service environment44 by a host computer. If the associated virtual area already has been instantiated, theserver network node42 sends provisioning instructions for establishing respective sessions between thevirtual presence apparatus12 and the client network nodes of the communicants who are present in the virtual area and for updating the appearance of the virtual area to include a graphical representation of thevirtual presence apparatus12 in the graphical interfaces displayed on the client network nodes. If the associated virtual area has not yet been instantiated, theserver network node42 instantiates the associated virtual area so that communicants operating respective client network nodes can access the virtual area.
• The provisioning instructions sent by theserver network node42 are used to establish communication sessions between the client network nodes and the virtual presence apparatus. In some examples, data is shared between the client network nodes and thevirtual presence apparatus12 as definition records over transport protocol sockets. The client network nodes and thevirtual presence apparatus12 receive content from each other through definition records that are received on content-specific channels on respective peer-to-peer sessions. Data is shared in accordance with a publish/subscribe model. A stream transport service on each of the client network nodes and thevirtual presence apparatus12 subscribes only to the data that are needed. To subscribe, the stream transport service negotiates a channel on a session that is established with another network node. The channel is negotiated by well-known GUID for theparticular area application48. Definition records are transmitted only when a subscriber exists on the other end of a transport protocol socket. Definition records that are received by the stream transport service are delivered to the subscribing ones of the local communication processes on arrival. Example of the structure and operation of the stream transport service and the data sharing communication sessions are described in U.S. patent application Ser. No. 12/825,512, filed Jun. 29, 2010.
• In some examples, thevirtual presence apparatus12 transmits the output data corresponding to the human perceptible stimulus in the physical space to the client network nodes in connection with the virtual area. In this process, thevirtual presence apparatus12 typically processes the output data and configures its communication interface to incorporate the output data into the output signal that is sent to a client network node. In some examples, the output signal includes at least one of the globally unique identifier of the virtual presence apparatus, an identifier of the virtual area, and optionally an identifier of a zone of the virtual area. The output signal typically is free of any communicant identifier (i.e., an identifier that identifies a particular communicant). In these examples, the virtual area (or the specified zone of the virtual area) serves as a termination point for one or more data streams that represent physical stimuli in a physical space occupied by thevirtual presence apparatus12, where the data streams are published by the virtual presence apparatus in the virtual area/zone, communicants who are present in the virtual area/zone respectively are able to subscribe to one or more of the data streams, and theserver network node42 provisions data stream connections for the data streams that are subscribed to by respective ones of the communicants who are present in the virtual area/zone.
• Thevirtual presence apparatus12 typically determines the input data from the input signal that is received through its communication interface from a respective client network node that is connected to the virtual area. The input signal typically includes at least one of a globally unique identifier of the respective client network node and an identifier of the virtual area. Thevirtual presence apparatus12 typically derives input data from the input signal and passes the input data to an output transducer, which generates human perceptible stimulus in the physical space.
• FIG. 6 shows an example of a method that is implemented by an example of theserver network node42 for administering communications between a virtual area and a physical space. In accordance with this method, theserver network node42 creates a persistent association between virtual presence apparatus in a physical space and a virtual area (FIG. 6, block100). The apparatus has an apparatus source of a respective data stream content type and an apparatus sink of a respective data stream content type. Theserver network node42 establishes a respective presence in the virtual area for a communicant operating a client network node connected to the virtual area (FIG. 6, block102). The client network node has a client sink that is complementary to the apparatus source and a client source that is complementary to the apparatus sink. Theserver network node42 administers a respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus in association with the virtual area, where each connection supports transmission of the respective data stream content type between the apparatus and the client network node (FIG. 6, block104).
• In some examples of the method ofFIG. 6, the association between thevirtual presence apparatus12 and the virtual area is independent of any particular communicant. Theserver network node42 typically receives a globally unique identifier of thevirtual presence apparatus12, and associates the identifier with the virtual area. In some examples, the virtual area includes multiple zones each of which supports establishment of a respective presence for one or more communicants and defines a respective persistent context for realtime communications between the client network nodes of communicants who are present in the zone. In some of these examples, theserver network node42 creates a persistent association between the physical presence apparatus and a respective one of the zones of the virtual area.
• Theserver network node42 establishes a respective presence in the zone for the virtual presence apparatus. In some examples, theserver network node42 establishes the presence for the virtual presence apparatus in response to receipt of a login request identifying the virtual presence apparatus. The virtual presence apparatus or a network node (e.g., a central hub or a client network node) that is hosting the virtual presence apparatus may generate the login request for the virtual presence apparatus. In some examples, theserver network node42 establishes a presence for both the virtual presence apparatus and a communicant in response to respective login requests that are sent by the same client network node. In some examples, in response to receipt of a login request that includes the identifier of the virtual presence apparatus, theserver network node42 initiates the virtual area to enable the virtual area to be communicant accessible.
• Theserver network node42 typically associates thevirtual presence apparatus12 with an object in the virtual area. Theserver network node42 typically creates an object that represents thevirtual presence apparatus12 in the virtual area. The object typically is associated with an interface for interacting with thevirtual presence apparatus12. In some examples, theserver network node42 associates the object with a graphical representation of thevirtual presence apparatus12. In some examples, the graphical representation of thevirtual presence apparatus12 includes a brand that is associated with the virtual presence apparatus. The brand may include a name, term, design, symbol, or any other feature that identifies a source (e.g., manufacturer or seller) of the virtual presence apparatus. Theserver network node42 transmits to each of the communicants who are present in the zone a respective specification of a visualization of graphical representations of the object and the avatars in the virtual area. The client network nodes use the specifications to display respective graphical representations of thevirtual presence apparatus12 and the communicants in spatial relation to a graphical representation of the virtual area.
• In some examples, the object representing the virtual presence apparatus is associated with a particular communicant and the visualization of the virtual area shows an association between a graphical representation of the object and the particular communicant. In some of these examples, the visualization of the virtual area shows the graphical representation of the object associated with a graphical representation of the avatar representing the particular communicant. For example, the virtual presence apparatus may be personal gear (e.g., a human interface device, such as a headset, or other personal device) that is carried or worn by the particular communicant, and the visualization may show a graphical representation of the gear as a decoration or embellishment on the graphical representation of the particular communicant's avatar (e.g., showing a graphical representation of a headset on the communicant's avatar). In some examples, the visualization of the virtual area shows the graphical representation of the object representing the virtual presence apparatus associated with a location in the virtual area that is assigned to the particular communicant. For example, the virtual presence apparatus may be personal gear (e.g., a personal printer, scanner, telephony device, or a memory resource of a personal computer) that is assigned or belongs to the particular communicant, and the visualization may show a graphical representation of the in a room (e.g., an office or personal room) of the virtual area that is assigned to the particular communicant. In some of these examples, theserver network node42 may determine the style used to represent the personal gear in the visualization based on configuration information received from the particular communicant (e.g., an indication that the graphical representation of the gear should be associated with the communicant's avatar or the communicant's designated default zone, such as the communicant's home zone or office) or automatically based on a predefined mapping between personal gear category types and presentation styles (e.g., headsets are represented as graphical decorations on the respective communicants' avatars, whereas hard drive of personal computers are represented as icons in the respective communicants' designated default zones).
• In some examples, theserver network node42 transmits to the client network node a specification of visual cues for displaying indications of respective states of a source of thevirtual presence apparatus12. Based on a determination that that the source of the virtual presence apparatus is in an active state, theserver network node42 transmits to the client network node a specification of a first visual cue, and based on a determination that the source of the virtual presence apparatus is in an inactive state, theserver network node42 transmits to the client network node a specification of a second visual cue that is different from the first visual cue. In some examples, the specifications of the first and second visual cues are provided in respective definition records.
• Theserver network node42 administers realtime communications between the respective network nodes of the communicants who are present in the zone and provisions at least one data stream connection between thevirtual presence apparatus12 and one or more of the network nodes of the communicants who are present in the zone. In some examples, theserver network node42 administers respective connections between each active pair of complementary sources and sinks of the client network node and the apparatus. These connections bridge communicant activity in the physical space into the virtual area and bridge communicant activity in the virtual area into the physical space. In some of these examples, theserver network node42 administers connections that relay data corresponding to communicant activity in the physical space from the source of thevirtual presence apparatus12 to the client network node. In some of these examples, theserver network node42 administers connections that relay data corresponding to communicant activity in the virtual area from the client network node to the sink of thevirtual presence apparatus12. In some examples, thevirtual presence apparatus12 publishes data streams of different data stream types, and theserver network node42 provisions the client network nodes to receive data streams of different data stream types that are published by the particular physical apparatus. In some examples, theserver network node42 provisions a data stream connection between a client network node and the virtual presence apparatus in response to a request from the client network node to subscribe to data published by the particular physical apparatus. In some examples, theserver network node42 provisions a data stream connection between a client network node of a particular communicant and the virtual presence apparatus automatically upon entry of the particular communicant into the zone.
• In some examples, the source of thevirtual presence apparatus12 corresponds to a transducer that transforms human perceptible stimulus that is broadcasted in the physical space into output data of the respective data stream content type. In some examples, the source of thevirtual presence apparatus12 corresponds to a transducer that transforms input data of the respective data stream content type into human perceptible stimulus that is broadcasted into the physical space. In some examples, the source of thevirtual presence apparatus12 includes a microphone and the sink of thevirtual presence apparatus12 includes a speaker. The microphone generates output voice data from human voice sound projected into the physical space. The speaker projects human voice sound into the physical space based on input voice data associated with the virtual area. In some of these examples, theserver network node42 administers connections that relay the output voice data from the apparatus to the client network node and that relay the input voice data from the client network node to the apparatus. In some examples, the source of thevirtual presence apparatus12 includes a camera that captures images of a scene in thephysical space14 and generates output image data from the captured images. In some of these examples, theserver network node42 administers a connection that relays the output image data from thevirtual presence apparatus12 to the client network node. In some examples, the sink of thevirtual presence apparatus12 includes a projector that projects images into the physical space. In some of these examples, theserver network node42 administers a connection that relays input control data for controlling the projecting from the client network node to thevirtual presence apparatus12. In some examples, the sink of thevirtual presence apparatus12 includes a laser pointer that projects a laser beam into the physical space. In some of these examples, theserver network node42 administers a connection that relays input control data for controlling the projecting of the laser beam from the client network node to thevirtual presence apparatus12.
• Thus, in some examples, based on communicant input in connection with the object representing thevirtual presence apparatus12, theserver network node42 administers a connection between an audio source of the client network node and an audio sink of thevirtual presence apparatus12. In some examples, based on communicant input in connection with the object, theserver network node42 administers a connection between an application sharing source of the client network node and an image projection sink of thevirtual presence apparatus12. In some examples, based on communicant input in connection with the object, theserver network node42 administers a connection between a laser pointer control source of the client network node and a laser pointer control sink of the apparatus.
• In some examples, thevirtual presence apparatus12 is located in a particular physical space, and theserver network node42 locates the object representing thevirtual presence apparatus12 in a particular one of the zones of the virtual area according to a mapping between the particular physical space and the particular zone. In some of these examples, the mapping associates an identifier of the physical space with an identifier of the particular zone, creating a persistent association between the particular physical space and the particular zone of the virtual area. In some of these examples, the mapping additionally associates an identifier of thephysical apparatus12 with the identifier of the physical space. In some examples, the visualization of the virtual area shows the particular zone with a label that connotes a name associated with the physical space.
• In some examples, theserver network node42 establishes a respective presence in the virtual area for a particular communicant based on a determination that the particular communicant is in thephysical space14. In some examples, theserver network node42 receives location data (e.g., Global Positioning System (GPS) data) that is associated with the particular communicant (e.g., by a GPS component of a mobile device, such as a mobile phone or other mobile communication device), and determines that the particular communicant is in the physical space based on comparison of the received location data with location data associated with the physical space. In some examples, theserver network node42 receives audio data from the source ofvirtual presence apparatus12, and associates the audio data with a communicant in the physical space based on comparison of the audio data with one or more voice data records associated with respective communicants. The voice records typically correspond to voiceprints (also referred to as voice templates or voice models) that are created from features that are extracted from the recorded speech of known communicants in accordance with a speaker recognition enrollment process. Each voiceprint is associated with the identity of a particular communicant. Theserver network node42 typically associates the audio data with the communicant in response to a determination that features extracted from the audio data correspond to the voiceprint previously associated with the communicant. In this way, theserver network node42 can automatically identify communicants who are in the physical space without requiring them to log into the networkinfrastructure service environment44 through respective client network nodes. Once a particular communicant in thephysical space14 has been identified, theserver network node42 can automatically establish a presence for that communicant in the virtual area associated with thevirtual presence apparatus12 and track utterances from that communicant in the audio data captured by the virtual presence apparatus such that visual cues indicative of the state of that communicant's voice (e.g., speaking or silent) can be presented in the spatial visualization of the virtual area that is displayed to the remote communicant on the remoteclient network node16.
• FIG. 7 shows an example of a method that is performed by an example of thecommunications application20 for communicating between a virtual area and a physical space. In accordance with this method, thecommunications application20 displays a graphical interface that includes a graphical representation of the virtual area that supports establishment of respective presences of communicants operating respective client network nodes, a graphical representation of each of the communicants who is present in the virtual area, and a graphical representation of an object associated with an apparatus (e.g., the virtual presence apparatus12) in the physical space (FIG. 7, block110). The apparatus has an apparatus sink that is complementary to the client source and an apparatus source that is complementary to the client sink. Thecommunications application20 establishes a respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus in association with the virtual area, where each connection supports transmission of the respective data stream content type between the apparatus and the client network node (FIG. 7, block112). Thecommunications application20 presents interaction controls associated with the object for interacting with communicants who are present in the physical space (FIG. 7, block114).
• In some examples of the method ofFIG. 7, the graphical representation of the virtual area corresponds to a virtualized representation of the physical space. In some of these examples, the virtualized representation connotes the real-world physical space. For example, the virtualized representation may have a virtual presentation that resembles one or more distinctive visual features of the real-world physical space or the virtualized representation may include a descriptive name or other label that is associated with the real-world physical space.
• In some examples, thecommunications application20 receives from a network service administering the virtual area a specification for displaying the graphical representation of the object in spatial relation to the graphical representation of the virtual area.
• In some examples, thecommunications application20 shows in the graphical interface indications of respective states of the apparatus source of thevirtual presence apparatus12 in connection with the graphical representation of the object. In some of these examples, the process of showing the state indications involves displaying a first visual cue when the virtual presence apparatus source is in an active state, and displaying a second visual cue that is different from the first visual cue when the virtual presence apparatus source is in an inactive state.
• In some examples, based on communicant input in connection with the object, thecommunications application20 establishes a connection between an audio source of the client network node and an audio sink of thevirtual presence apparatus12.
• In some examples, based on communicant input in connection with the object, thecommunications application20 establishes a connection between an application sharing source of the client network node and an image projection sink of thevirtual presence apparatus12.
• In some examples, based on communicant input in connection with the object, thecommunications application20 establishes a connection between a laser pointer control source of the client network node and a laser pointer control sink of thevirtual presence apparatus12.
• In some examples, the source of thevirtual presence apparatus12 includes a microphone and the sink of thevirtual presence apparatus12 includes a speaker. The microphone generates output voice data from human voice sound projected into the physical space, the speaker projects human voice sound into the physical space from input voice data associated with the virtual area, and thecommunications application20 establishes connections that relay the output voice data from thevirtual presence apparatus12 to the client network node and that relay the input voice data from the client network node to thevirtual presence apparatus12.
• In some examples, the source of thevirtual presence apparatus12 includes a camera that captures images of a scene in the physical space and generates output image data from the captured images, and thecommunications application20 establishes a connection that relays the output image data from thevirtual presence apparatus12 to the client network node.
• In some examples, the sink of thevirtual presence apparatus12 includes a projector that projects images into the physical space, and thecommunications application20 establishes a connection that relays input control data for controlling the projecting from the client network node to thevirtual presence apparatus12.
• In some examples, the sink of thevirtual presence apparatus12 includes a laser pointer that projects a laser beam into the physical space, and thecommunications application20 establishes a connection that relays input control data for controlling the projecting of the laser beam from the client network node to thevirtual presence apparatus12.
• FIG. 8 shows an example of agraphical interface120 that is generated by thecommunications application20 on a client network node (e.g., client node16) for interfacing a user with an example122 of thevirtual presence apparatus12 in thephysical space14.
• Thegraphical interface120 includes atoolbar124 and aviewer panel126. Thetoolbar124 includes aheadphone control128 for toggling on and off the local speakers of the client network node, amicrophone control130 for toggling on and off the local microphone of the client network node, and aview screen button132 for setting theviewer panel126 to content being shared in connection with theview screen object60. The user also may select theview screen object60 to initiate an application sharing session in thevirtual area46. Theviewer panel126 typically shows communicant selectable content being rendered by the client network node. Examples of such content include a spatial visualization of the virtual area46 (currently shown) and application content (e.g., web service content rendered by a web browser application such as Microsoft® Internet Explorer®, or document content being rendered by a document processing application such as Microsoft® Word® or Power Point® software applications).
• In the example shown inFIG. 8, thevirtual presence apparatus122 is a virtual area enabled speakerphone, which is represented by aspeakerphone object138 in the spatial visualization of thevirtual area46 that is shown in theviewer panel126. Thevirtual presence apparatus122 includes a microphone that generates output voice data from human voice sounds projected into thephysical space14 and a speaker that projects human voice sounds into thephysical space14 based on input voice data associated with the virtual area. The “on” or “off” state of the speakerphone microphone is depicted in the spatial visualization of the virtual area by the presence or absence of a series ofconcentric circles140 that dynamically radiate away from thespeakerphone object138 in a series of expanding waves. When the microphone is on, the radiatingconcentric circles140 are present and, when the microphone is off, the radiatingconcentric circles140 are absent. In addition to or alternatively, the current activity state of the speakerphone microphone channel is indicated by a dynamic visualization that lightens and darkens thespeakerphone object138 in realtime to reflect the presence or absence of audio data on the speakerphone microphone channel. Thus, the user can determine when a communicant in thephysical space14 is speaking by the “blinking” of the coloration of thespeakerphone object138.
• FIG. 9 shows an example of agraphical interface150 that is generated by thecommunications application20 on a client network node (e.g., client node16) for interfacing a user with an example152 of thevirtual presence apparatus12 in thephysical space14.
• Thegraphical interface150 includes thetoolbar124 and theviewer panel126 of thegraphical interface120 shown inFIG. 8. Thetoolbar124 includes theheadphone control128 for toggling on and off the local speakers of the client network node, themicrophone control130 for toggling on and off the local microphone of the client network node, and aview screen button132 for setting theviewer panel126 to content being shared in connection with theview screen object60. The user may select theview screen object60 to initiate an application sharing session in thevirtual area46. Theviewer panel126 typically shows communicant selectable content that is rendered by the client network node.
• In the example shown inFIG. 9, thevirtual presence apparatus152 is a virtual area enabled device that integrates speakerphone and digital projector functionalities. Thevirtual presence apparatus152 includes a microphone that generates output voice data from human voice sounds projected into thephysical space14, a speaker that projects human voice sounds into thephysical space14 based on input voice data associated with the virtual area, and a projector that projects light into thephysical space14 responsive to input data transmitted by the client network node in connection with thevirtual area46.
• Thevirtual presence apparatus152 is represented by aprojector object154 in the spatial visualization of thevirtual area46 that is shown in theviewer panel126. In this example, when the user selects theprojector object154, thecommunications application20 modifies thegraphical interface150 to include aShare button134 and a Stop button136 in thetool bar124, and sets theviewer panel126 to display the contents of an application being shared. The user initiates an application sharing session in thephysical space14 by selecting theShare button134. In response to the selection of theshare button134, thecommunications application20 provides an interface that enables the user to select an application to share (e.g., Microsoft® PowerPoint®), sets theviewer panel126 to display the contents being rendered by the selected application, and streams screen share data to thevirtual presence apparatus152, which projects the screen share data onto the real-world view screen34 in thephysical space14. Examples of systems and methods of generating and streaming screen share data are described in U.S. patent application Ser. No. 12/418,270, filed Apr. 3, 2009. The user can terminate the application sharing session in thephysical space14 by selecting the Stop button136.
• FIG. 10 shows an example of agraphical interface160 that is generated by thecommunications application20 on a client network node (e.g., client node16) for interfacing a user with an example162 of thevirtual presence apparatus12 in thephysical space14. In this example, thecommunicant34 is giving a presentation on awhite board158 in thephysical space14.
• Thegraphical interface160 includes thetoolbar124 and theviewer panel126 of thegraphical interface120 shown inFIG. 8. Thetoolbar124 includes theheadphone control128 for toggling on and off the local speakers of the client network node, themicrophone control130 for toggling on and off the local microphone of the client network node, and aview screen button132 for setting theviewer panel126 to content being shared in connection with theview screen object60. The user may select theview screen object60 to initiate an application sharing session in thevirtual area46. Theviewer panel126 typically shows communicant selectable content being rendered by the client network node.
• In the example shown inFIG. 10, thevirtual presence apparatus162 is a virtual area enabled device that integrates a speakerphone, a digital projector, and a camera. The speakerphone includes a microphone that generates output voice data from human voice sounds projected into thephysical space14, and a speaker that projects human voice sounds into thephysical space14 based on input voice data associated with the virtual area. The projector projects light (e.g., images, shapes, lines, and spots) into thephysical space14 responsive to input data transmitted by the client network node in connection with thevirtual area46. In some examples, the projector is a digital image projector. In other examples, the projector is a remote-controlled laser pointer. The camera captures images of a scene in the physical space14 (e.g., images of the whiteboard158) and generates output image data from the captured images. The camera may be implemented by any type of imaging device that is capable of capturing one-dimensional or two-dimensional images of a scene. The camera typically is a digital video camera.
• Thevirtual presence apparatus162 is represented by a projector-camera object164 in the spatial visualization of thevirtual area46 that is shown in theviewer panel126. In this example, when the user selects the projector-camera object164, thecommunications application20 modifies thegraphical interface150 to include aShare button134 and a Stop button136 in thetool bar124, and sets theviewer panel126 to display the images captured by thevirtual presence apparatus162 in thephysical space14. The user initiates a sharing session in thephysical space14 by selecting theShare button134. In response to the selection of theShare button134, thecommunications application20 sets theviewer panel126 to display the images captured by thevirtual presence apparatus162, provides an interface that enables the user to provide inputs in relation to the images displayed in the viewer panel (e.g., superimpose graphical content, such as predesigned or hand drawn images or comments, onto the images), and streams data describing the inputs to thevirtual presence apparatus162, which projects the streamed data onto thewhiteboard158 in thephysical space14. The user can terminate the sharing session in thephysical space14 by selecting the Stop button136.
• In the illustrated example, thecommunications application20 provides drawings tools (e.g., the pencil tool166) that allow the user to superimpose lines, shapes (e.g., the ellipse168), and other graphical content onto the image of theview screen34 captured by the camera component of thevirtual presence apparatus162.
• In examples in which the projector component of thevirtual presence apparatus162 is a digital projector, thecommunications application20 may stream data describing the user inputs to thevirtual presence apparatus162. In examples in which the projector is a remote-controlled laser pointer, thecommunications application20 may convert the user inputs into control data for controlling the movement of the remote-controlled laser pointer in thephysical space14. In this way, the user can interact with the communicants in thephysical space14 in a substantive way. For example, the user can provide comments or other visual indications that highlight or direct a viewer's attention to specific parts of the presentation being given by thecommunicant32 in connection with thewhite board158. In some examples, thegraphical interface160 includes additional controls for streaming application sharing data from the client network node to thevirtual presence apparatus162 for projection onto thewhiteboard158 or other surface in thevirtual space14, as described above in connection with the example shown inFIG. 9.
• FIG. 11 shows an example of agraphical interface170 that is generated by thecommunications application20 on a client network node for interfacing a user with an example172 of thevirtual presence apparatus12 in thephysical space14. In this example, thecommunicant34 is giving a presentation on awhite board158 in thephysical space14.
• Thegraphical interface170 includes thetoolbar124 and theviewer panel126 of thegraphical interface120 shown inFIG. 8. Thetoolbar124 includes theheadphone control128 for toggling on and off the local speakers of the client network node, themicrophone control130 for toggling on and off the local microphone of the client network node, and aview screen button132 for setting theviewer panel126 to content being shared in connection with theview screen object60. The user may select theview screen object60 to initiate an application sharing session in thevirtual area46. Theviewer panel126 typically shows communicant selectable content being rendered by the client network node.
• In the example shown inFIG. 11, thevirtual presence apparatus162 is a virtual area enabled device that integrates a speakerphone and a camera. The speakerphone includes a microphone that generates output voice data from human voice sounds projected into thephysical space14, and a speaker that projects human voice sounds into thephysical space14 based on input voice data associated with the virtual area. The camera captures images of a scene in thephysical space14 and generates output image data from the captured images. The camera may be implemented by any type of imaging device that is capable of capturing one-dimensional or two-dimensional images of a scene. The camera typically is a digital video camera.
• The virtual presence apparatus172 is represented by acamera object174 in the spatial visualization of thevirtual area46 that is shown in theviewer panel126. In this example, when the user selects thecamera object174, thecommunications application20 modifies thegraphical interface150 to include aView button176 and aStop button178 in thetool bar124, and sets theviewer panel126 to display the images captured by thevirtual presence apparatus162 in thephysical space14. The user initiates a viewing session in thephysical space14 by selecting theView button176. In response to the selection of theView button176, thecommunications application20 sets theviewer panel126 to display the images captured by the virtual presence apparatus172, provides aninterface180 that enables the user to control the view of the physical space that is captured by the camera component of the virtual presence apparatus, and streams data describing the control inputs to the virtual presence apparatus172, which moves the camera based on the streamed data. In the illustrated example, thecommunications application20 provides anavigation control tool180 that allows the user to control the pan and tilt of the camera component of the virtual presence apparatus172. In this way, the remote communicant can interact with thephysical space14 in a substantive way (e.g., see different views of the persons and activities in the physical space14). The user can terminate the sharing session in thephysical space14 by selecting theStop button178.
• In some examples, different elements of the graphical interfaces described above in connection with the examples shown inFIGS. 8-11 are incorporated into a single graphical interface that may be used to interact with thevirtual presence apparatus164, which integrates a speakerphone, a digital projector, and a camera. In these examples, the graphical interface provides independent control over the respective functionalities of the speakerphone, the digital projector, and camera to enable application sharing, image projection of comments and other annotations, and camera viewing modes of operation.
• In some examples, thevirtual environment creator18 enhances the immersive connections between virtual area locations (e.g., zones) and physical spaces by creating persistent associations between the virtual area locations and the respective physical spaces. These persistent associations typically are stored in a table or other data structure that maps each real-world location to a respective zone. In some of these examples, thevirtual environment creator18 reinforces these associations with visual cues in the visualizations of the virtual area locations that connote the real-world physical spaces (e.g., by having a virtual presentation that resembles one or more distinctive visual features of the real-world physical space or by including a descriptive name or other label that is associated with the real-world physical space).
• FIG. 12 shows an example of a two-dimensional visualization of a virtual area200 (“Sococo HQ”). The Sococo HQ virtual area includes alobby202, aMain conference room204, aWest Conference room206, anEast Conference room208, aWest Nook zone210, anEast Nook zone212, aCourtyard zone214, and sixteen offices. The conference rooms204-208 include respective viewscreen objects216-230, table objects232,234, and236, and objects representing respectivevirtual presence apparatus238,240,242 and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same conference room. Each of the offices includes respective viewscreen objects (not shown) and a respective telephony object (not shown) and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same office. Each of the telephony objects supports shared dial-in and dial-out telephony communications as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011, and communicants interacting with the telephony objects are represented by avatars decorated with a graphical representation of a telephone (see, e.g., the avatar215 in Carl's Office). Each of theWest Nook210,East Nook212, andLobby202 zones respectively supports realtime audio and chat communications between the network nodes in the same zone.
• In some examples, the conference rooms204-208 are associated with different real-world physical spaces. The different real-world physical spaces may be physically connected to or proximate one another (e.g., rooms connected by a common structure, such as rooms in an office building, or disconnected rooms of related co-located structures, such as rooms in a distributed office building complex) or they may be physically remote from one another (e.g., rooms in separate and distinct real-world office buildings, which may be in the same or different geographic regions). Thevirtual environment creator18 reinforces these associations with visual cues in the visualizations of the virtual area locations that connote the corresponding real-world physical spaces. In the example shown inFIG. 12, each of the virtual conference rooms204-208 is labeled with a respective name (e.g., Main, West Conference, and East Conference) that corresponds to the name that is used to identify the corresponding real-world physical space. In addition, virtual presentations of the virtual conference rooms204-208 include respective features (e.g., the number and placement of virtual view screens216-230,virtual plants240,242 and virtual artwork244) that correspond to distinctive visual features of the associated real-world physical spaces. The resulting visualization of thevirtual area200 allows a user to see multiple concurrent independent conversations and other interactions that are occurring in different physical spaces in a single view in which the interactions are organized according to a spatial metaphor that allows the user to quickly learn who is meeting with whom and the contexts of those meetings (as defined by the zones in which the meetings are occurring). In addition, the objects238-242 in the virtual conference rooms204-208 provide interfaces for communicants in thevirtual area200 to interact with the associated virtual presence apparatus and thereby be bridged into the corresponding physical spaces.
• FIG. 13 shows an example of a method by which theserver network node42 manages communications between virtual area zones and multiple respective real-world locations via respective physical apparatus. In accordance with this method, theserver network node42 administers zones of one or more virtual areas in a virtual communications environment (FIG. 13, block190). Each of respective ones of the zones defines a respective persistent context for realtime communications between client network nodes of respective communicants who are present in the zone. In the process of administering the zones, theserver network node42 administers realtime communications between the respective network nodes of co-present communicants in respective ones of the zones. For each of multiple physical apparatus in respective real-world locations, theserver network node42 establishes a respective presence for the physical apparatus in a respective one of the zones based on mappings between the respective real-world location and the respective zone, and creates a respective object that represents the physical apparatus in the respective zone and is associated with a respective interface for communicant interaction with the physical apparatus (FIG. 13, block192). Theserver network node42 transmits to each of one or more of the respective client network nodes a respective specification of a visualization of a spatial layout of the zones, graphical representations of the objects in their respective zones of presence, and graphical representations of avatars representing communicants in their respective zones of presence (FIG. 13, block194). Theserver network node42 provisions respective data stream connections between respective ones of the physical apparatus and respective ones of the client network nodes (FIG. 13, block196).
• In some examples, theserver network node42 establishes the respective presence of each of respective ones of the physical apparatus in response to receipt of a respective login request that is generated by and identifies the respective physical apparatus.
• For each of one or more of the physical apparatus, theserver network node42 publishes in the respective zone of presence of the physical apparatus one or more physical space data streams that include respective representations of human perceptible physical stimuli in the respective real-world location. In the process of provisioning the respective data stream connections, theserver network node42 provisions data stream connections for transmitting respective ones of the published physical space data streams to respective ones of the client network nodes for transforming into human perceptible stimuli. Each of one or more of the client network nodes typically publishes one or more respective client data streams. In this case, theserver network node42 provisions data stream connections for transmitting respective ones of the published client data streams to respective ones of the client network nodes for transforming into human perceptible stimuli, and transmitting respective ones of the published client data streams to respective ones of the physical apparatus for transforming into human perceptible stimuli in the respective real-world locations.
• In some examples, a particular one of the physical apparatus is operable to perform a respective function in its respective real-world location in response to data transmitted by a particular one of the network nodes on a respective one of the data stream connections that is provisioned based on a request from the particular network node referencing the object representing the particular physical apparatus. In some examples, the particular physical apparatus sends notifications of events relating to the respective function that is performable by the particular physical apparatus. Based on a notification of an event relating to the respective function that is performable by the particular physical apparatus, theserver network node42 sends a notification of the event to the particular one of the client network nodes of a respective one of the communicants. For example, the particular physical apparatus may include a printer, in which case theserver network node42 may send to the particular network node a notification that a document has been printed. The particular physical apparatus may include a facsimile machine, in which case theserver network node42 may send to the particular network node a notification of an incoming facsimile. The particular physical apparatus may include a telephony device, in which case theserver network node42 may send to the particular network node a notification of an incoming telephone call.
• Theserver network node42 may provision a variety of different data stream connections between respective ones of the client network nodes and the physical apparatus. For example, the particular physical apparatus may include a printer and, based on a request from a particular client network node referencing the object representing the particular physical apparatus, theserver network node42 may provision at least one data stream connection for the particular network node to print a document. The particular physical apparatus may include a facsimile machine and, based on a request from a particular client network node referencing the object representing the particular physical apparatus, theserver network node42 may provision at least one data stream connection for the particular network node to one of send a facsimile and receive a facsimile. The particular physical apparatus may include a telephony device and, based on a request from a particular client network node referencing the object representing the particular physical apparatus, theserver network node42 may provision at least one data stream connection for the particular network node to one of place an outgoing telephone call and receive an incoming telephone call.
• FIG. 14 shows an example of anetwork communications environment300 that includes network resources that may be connected over anetwork302 to an example of aclient network node304 by thevirtual environment creator18 according tomappings305 between the network resources and zones of one or more virtual areas. Thevirtual environment creator18 sends a specification to theclient network node304 for generating an example of agraphical user interface306 that includes a spatial visualization that partitions the network resources into zones according to themappings305. In this example, the visualization shows a first zone344 (i.e., a Work Office zone) that is mapped to a real-world work office314 of a user (Ed in this example) and a second zone346 (i.e., a Home Office zone) that is mapped to the user's real-world home office322. In some examples, theWork Office zone344 and theHome Office zone346 are zones of a common virtual area (e.g., the user's Business virtual area). In other examples, theWork Office zone344 and theHome Office zone346 are zones of different virtual areas (e.g., a Work virtual area and a Home virtual area).
• In the example shown inFIG. 14, the network resources include: afacsimile machine308, a telephony device310 (e.g., a SIP phone), and anetwork video camera312 that are located in the user'swork office314; and aprinter316, atelephony device318, and ascanner320 that are located in the user'shome office322.
• Thegraphical user interface306 includes atoolbar326, aviewer panel328 of the type described above in connection withFIGS. 8-11, apeople panel330, and achat panel331.
• Theviewer panel328 includes a canvas area for displaying visual content. For example, theviewer panel328 displays virtual area visualizations that are rendered by the virtual area enabledcommunications application20, network resource content that is rendered by a web browser application such as Microsoft® Internet Explorer®, application sharing content that is being shared by the user or another communicant in the virtual area, and visual content received from virtual presence apparatus in the respective real-world locations that are linked to virtual area zones that are associated with the user. In the illustrated example, theviewer panel328 is operating in a map view mode that shows respective visualizations of theWork Office zone344 and theHome Office zone346. The visualization of theWork Office zone344 includes graphical representations of viewscreen objects338,340, a table object354, atelephony object356, (on the left-hand side from top to bottom) objects357 that respectively represent thefacsimile machine308, thephone310, and thecamera312 in the real-world work office314, and graphical representations of the communicants who are present in the virtualWork Office zone344. The visualization of theHome Office zone346 includes graphical representations of theviewscreen object342, a table object358, atelephony object360, (on the left-hand side from top to bottom) objects361 that respectively represent theprinter316, thephone318, and thescanner320 in the real-world home office322, and graphical representations of the communicants who are present in the virtualHome Office zone346. Each of the telephony objects356-360 supports shared dial-in and dial-out telephony communications with one or more public switched telephony (PSTN)devices303 over aPSTN305, as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011.
• Thetoolbar326 includes aheadphone control332 for toggling on and off the local speakers of the client network node, amicrophone control334 for toggling on and off the local microphone of the client network node, and one or moreview screen buttons336 for setting theviewer panel328 to show content being shared in connection with respective view screen objects338,340,342 in the user's current zone of presence (e.g., theWork Office zone344 or the Home Office zone346).
• Thepeople panel330 depicts the realtime availabilities and activities of some of or all the user's contacts across the different communication contexts defined by theWork Office zone344 and theHome Office zone346. In the example shown inFIG. 14, thepeople panel330 shows Ed′ contacts segmented into aWork Office section348, a Home Office section350, and aContacts section352. TheWork Office section348 shows graphical representations, respective states, and realtime activities of the communicants who are present in the Work Office zone344 (i.e., Ed, Paul, and David); the Home Office section350 shows graphical representations, respective states, and realtime activities of the communicants who are present in the Home Office zone346 (i.e., Josh and Matt); and theContacts section352 shows all or a selected portion of Ed's contacts who are not represented in any of theother sections348,350. Examples of thepeople panel330 are described in U.S. patent application Ser. No. 13/209,812, filed Aug. 15, 2011, and U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
• Thechat panel331 shows a chat interface for a persistent virtual chat area for interactions occurring in connection with the user's current zone of presence. In the example shown inFIG. 14, the user (Ed) is present in theWork Office zone344; therefore, thechat panel331 shows the persistent virtual chat area for text chat and other interactions occurring in theWork Office zone344. Examples of thechat panel331 are described in U.S. patent application Ser. No. 13/209,812, filed Aug. 15, 2011, and U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
• Thevirtual environment creator18 manages communications between communicants who are present in the virtual Work Office andHome Office zones344,346 and the physical apparatus308-312,316-320 in their respective real-world locations in accordance with themappings305 between thezones344,346 and the physical apparatus308-312,316-320. In this way, communicants in the virtualWork Office zone344 are able to interact with any of thefacsimile machine308, thephone310, and thecamera312 in the real-world Work Office314 via the interfaces provided in connection with thegraphical representations357 of those apparatus in the visualization of the virtualWork Office zone344; similarly, communicants in the virtualHome Office zone344 are able to interact with any of theprinter316, thephone318, and thescanner320 in the real-world Home Office322 via the interfaces provided in connection with the graphical representations361 of those apparatus in the visualization of the virtualHome Office zone346.
• In addition, thevirtual environment creator18 bridges real-world notifications (e.g., notifications of events, alerts, and the like) that are generated by the physical apparatus308-312,316-320 in their respective real-world locations314,322 into thevirtual zones314,322 and bridges responses to those notifications received in connection with thevirtual zones314,322 into the physical real-world locations314,322 of the physical apparatus308-312,316-320. For example, in the illustrated example, thefacsimile machine308 generates in the real-world Work Office314 a notification that a facsimile was received from a particular fax number. Thefacsimile machine308 also sends a fax receipt notification to thevirtual environment creator18. Thevirtual environment creator18 relays thefax receipt notification370 to the user and to other communicants who are present in the virtualWork Office zone344. The user may click theView Fax button372 in the notification window to cause the received facsimile to be displayed in theviewer panel328. In another example, thephone318 generates in the real-world Home Office322 a notification of an incoming call from a particular phone number. Thephone318 also sends an incoming call notification to thevirtual environment creator18. Thevirtual environment creator18 relays the incomingcall receipt notification374 to the user and to other communicants who are present in the virtualHome Office zone346. The user may click theAnswer Call button376 in the notification window to cause the user's headset to be connected to receive the incoming call.
• In addition to providing notifications of events, alerts and the like, thevirtual environment creator18 also provides visual cues indicating the states and realtime activities of the real-world physical apparatus308-312,316-320. In some examples, the “on” or “off” states of the real-world physical apparatus308-312,316-320 are indicated by having two different presentations of their respectivegraphical representations357,361 (e.g., a first or brighter coloration when a physical apparatus in turned-on, and a darker or dimmed coloration when the physical apparatus is turned-off). In some examples, the current activity states of the real-world physical apparatus308-312,316-320 are indicated by having two different presentations of their respectivegraphical representations357,361 (e.g., using a static graphical representation of a physical apparatus when the physical apparatus current is inactive, and using a dynamic graphical representation—e.g., a blinking of the coloration of the graphical representation—when the physical apparatus currently is active).
• In these ways, the network resources in the real-world Work Office and Home Office area available both physically (via their respective physical interfaces in their respective real-world locations) and virtually (via their respective virtual interfaces in their respective virtual locations). In addition, the presentation of the virtual representations of the physical apparatus according to the spatial metaphor shown inFIG. 14 provides a way for users to organize their network resources that is particularly more intuitive and effective than traditional non-spatial directory-based visualizations of network resources.
• In some examples, thevirtual environment creator18 administers virtual areas based on signals received from intelligent personal gear that is associated with communicants. In these examples, the personal gear is able to infer information about a communicant's state (e.g., a headset is on the communicant's head, the communicant is proximate his client network node, and the communicant is located in a particular physical space), determine state event changes based on those inferences, and report those state event changes to thevirtual environment creator18, which reflects them in the virtual area representation.
• In some of these examples, theserver network node42 administers a virtual area in a virtual communications environment. The virtual area includes one or more zones, where each zone defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone. For each zone, the server network node administers realtime communications between the respective network nodes of communicants who are co-present in the zone. For each of respective ones of the communicants who are present in the virtual area, theserver network node42 transmits a respective specification of a visualization of the virtual area that includes graphical representations of the one or more zones and avatars that respectively represent the communicants in the one or more zones in which they respectively have presence. From sensing apparatus co-located with a particular one of the communicants in a physical space, theserver network node42 receives state information describing information relating to a physical state of the particular communicant. Based on the state information, the server network node updates the specification of the visualization of the virtual area and the avatars and transmits the updated specification to each of respective ones of the communicants who are present in the virtual area.
• In some examples, the state information describes the current real-world location of the particular communication, and in the process of updating the visualization specification theserver network node42 provides an indication of the current real-world location of the particular communicant in the visualization. In some of these examples, the process of updating the visualization specification includes locating the avatar representing the particular communicant in a zone of the virtual area associated with the current real-world location of the particular communicant. In some of these examples, the process of updating the visualization specification includes providing a descriptive label indicative of the current real-world location of the particular communicant in association with the graphical representation of the avatar representing the particular communicant.
• In some examples, the state information describes the state of the particular communicant in relation to a physical device that is associated with the particular communicant, and theserver network node42 updates the graphical representation of the avatar representing the particular communicant based on the state of the particular communicant in relation to the physical device. In some of these examples, the physical device is a headset. Based on a determination that the state information indicates that the particular communicant is wearing the headset, theserver network node42 includes a graphical representation of a headset with the graphical representation of the avatar of the particular communicant.
• In some examples, the state information describes a physical relationship between the particular communicant and another one of the communicants who present in the virtual area. This information may be obtained from a variety of different detection apparatus that are able to detect when the communicants are in the same physical space or when one communicant is within a certain distance of the other communicant. In some of these examples, theserver network node42 updates the visualization specification by updating the graphical representations of the avatars of the particular communicant and the other communicant to reflect the physical relationship between the particular communicant and the other communicant. For example, based on a determination that the state information indicates that the particular communicant and the other communicant are co-located in a shared real-world location, theserver network node42 may include respective indications that the particular communicant and the other communicant are physically co-located with the graphical representations of the avatars of the particular communicant and the other communicant.
III. Conclusion
• Other embodiments are within the scope of the claims.

Claims (57)

1. A method, comprising:

administering a virtual area in a virtual communications environment, wherein the virtual area comprises a zone that defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone;

establishing a respective presence in the zone for each of a particular physical apparatus and a particular communicant associated with a particular network node;

creating an object representing the particular physical apparatus in the virtual area and an avatar representing the particular communicant in the virtual area, wherein the object is associated with an interface for interacting with the particular physical apparatus;

provisioning at least one data stream connection between the particular physical apparatus and the particular network node.

2. The method ofclaim 1, further comprising:

establishing a respective presence for each of one or more other communicants in the zone;

for each of the other communicants, creating a respective avatar representing the other communicant in the virtual area; and

administering realtime communications between the respective network nodes of the communicants present in the zone.

3. The method ofclaim 2, further comprising transmitting to the particular communicant and each of the other communicants a respective specification of a visualization of graphical representations of the object and the avatars in the virtual area.

4. The method ofclaim 3, wherein the graphical representation of the object comprises a brand associated with the particular physical apparatus.

5. The method ofclaim 1, wherein the establishing of the presence for the particular physical apparatus is based on a globally unique identifier of the particular physical apparatus.

6. The method ofclaim 1, wherein the establishing of the presence for the particular physical apparatus is performed responsive to receipt of a login request identifying the particular physical apparatus.

7. The method ofclaim 6, wherein the login request is generated by the particular physical apparatus.

8. The method ofclaim 1, wherein the particular physical apparatus comprises at least one of a printer, a scanner, a facsimile machine, and a telephony device.

9. The method ofclaim 1, wherein the particular physical apparatus is associated with the particular network node.

10. The method ofclaim 9, wherein the particular physical apparatus is an integral component of the particular network node.

11. The method ofclaim 10, wherein the particular physical apparatus is a memory component of the particular network node.

12. The method ofclaim 9, wherein the particular physical apparatus is a peripheral device linked to the particular network node.

13. The method ofclaim 9, wherein the establishing of the presence for the particular physical apparatus is performed responsive to receipt of a login request from the particular network node.

14. The method ofclaim 13, wherein the establishing of the presence for the particular communicant is performed responsive to receipt of a login request from the particular network node.

15. The method ofclaim 9, wherein the creating comprises associating the object with the particular communicant, and further comprising generating a specification of a visualization of the virtual area showing an association between a graphical representation of the object and the particular communicant, and transmitting the specification to the particular network node.

16. The method ofclaim 15, wherein the visualization of the virtual area shows the graphical representation of the object associated with a graphical representation of the avatar representing the particular communicant.

17. The method ofclaim 15, wherein the visualization of the virtual area shows the graphical representation of the object associated with a location in the virtual area that is assigned to the particular communicant.

18. The method ofclaim 1, wherein the zone of the virtual area serves as a termination point for one or more data streams that represent physical stimuli in a physical space occupied by the particular physical apparatus, the data streams are published by the particular physical apparatus in the zone, communicants who are present in the zone respectively are able to subscribe to one or more of the data streams, and the provisioning comprises provisioning data stream connections for the data streams that are subscribed to by respective ones of the communicants who are present in the zone.

19. The method ofclaim 1, wherein the provisioning comprises provisioning an audio data stream connection for transmitting audio data from a microphone source of the particular physical apparatus to a sound rendering sink of the particular network node, and provisioning an audio data stream connection for transmitting audio data from a microphone source of the particular network node to a sound rendering sink of the particular physical apparatus.

20. The method ofclaim 1, wherein the provisioning comprises provisioning an image data stream connection for transmitting image data from a camera source of the particular physical apparatus to an image renderer sink associated with an image output device of the particular network node.

21. The method ofclaim 1, wherein the provisioning comprises provisioning an image data stream connection for transmitting image data from the particular network node to an image renderer sink associated with an image output device of the particular physical apparatus.

22. The method ofclaim 1, wherein the provisioning comprises provisioning a control data stream connection for transmitting control data from the particular network node to a camera control sink associated with a camera of the particular physical apparatus.

23. The method ofclaim 1, wherein the provisioning comprises provisioning a control data stream connection for transmitting control data from the particular network node to a pointer control sink associated with a pointer of the particular physical apparatus.

24. The method ofclaim 1, wherein the provisioning is performed in response to a request from the particular network node to subscribe to data published by the particular physical apparatus.

25. The method ofclaim 1, wherein the provisioning is performed automatically upon entry of the particular communicant into the zone.

26. The method ofclaim 1, wherein the particular physical apparatus publishes data streams of different data stream types, and the provisioning comprises provisioning the particular network node to receive data streams of different data stream types that are published by the particular physical apparatus.

27. The method ofclaim 1, further comprising:

establishing a respective presence in the zone for each of one or more other particular physical apparatus;

for each of the other particular physical apparatus, creating a respective other object representing the other particular physical apparatus in the virtual area; and

for each of the other particular physical apparatus, provisioning at least one data stream connection between the other particular physical apparatus and the particular network node.

28. The method ofclaim 27, wherein the provisioning is performed in response to one or more requests from the particular network node to subscribe to data published by one or more of the other particular physical apparatus.

29. The method ofclaim 1, wherein the virtual area comprises multiple zones, and each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone.

30. The method ofclaim 29, further comprising transmitting to the particular communicant and other communicants present in the virtual area a respective specification of a visualization of a spatial layout of the zones of the virtual area and graphical representations of the object and avatars representing communicants in respective ones of the zones of the virtual area.

31. The method ofclaim 30, wherein the particular physical apparatus is located in a physical space, and further comprising locating the object representing the particular physical apparatus in a particular one of the zones of the virtual area according to a mapping between the physical space and the particular zone.

32. The method ofclaim 31, wherein the mapping associates an identifier of the physical space with an identifier of the particular zone.

33. The method ofclaim 32, wherein the mapping additionally associates an identifier of the particular physical apparatus with the identifier of the physical space.

34. The method ofclaim 32, further comprising, in the visualization, labeling the particular zone with a label that connotes a name associated with the physical space.

35. A method, comprising:

transforming human perceptible physical stimuli in a physical space into physical space data streams of different respective data stream types;

publishing respective ones of the physical space data streams in a zone of a virtual area in a virtual communications environment, wherein the zone defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone;

establishing a respective presence in the zone for each of multiple communicants associated with respective client network nodes, each of one or more of the client network nodes publishing one or more respective client data streams;

provisioning data stream connections for transmitting respective ones of the published physical space data streams to respective ones of the client network nodes, transmitting respective ones of the published client data streams to respective ones of the client network nodes, and transmitting respective ones of the published client data streams to the physical space; and

transforming the published client data streams transmitted to the physical space into human perceptible physical stimuli in the physical space.

36. A method, comprising by a computer system:

in a zone of a virtual area in a virtual communications environment, publishing physical space data streams of different respective data stream types comprising respective representations of human perceptible physical stimuli in a physical space, wherein the zone defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone;

establishing a respective presence in the zone for each of multiple communicants associated with respective client network nodes, each of one or more of the client network nodes publishing one or more respective client data streams;

provisioning data stream connections for transmitting respective ones of the published physical space data streams to respective ones of the client network nodes for transforming into human perceptible stimuli, transmitting respective ones of the published client data streams to respective ones of the client network nodes for transforming into human perceptible stimuli, and transmitting respective ones of the published client data streams into the physical space for transforming into human perceptible stimuli in the physical space.

37. The method ofclaim 36, further comprising transforming human perceptible physical stimuli in a physical space into physical space data streams of different respective data stream types, and transforming the published client data streams transmitted into the physical space into human perceptible physical stimuli in the physical space.

38. A method, comprising:

administering zones of one or more virtual areas in a virtual communications environment, wherein each of respective ones of the zones defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone, and the administering comprises administering realtime communications between the respective network nodes of co-present communicants in respective ones of the zones;

for each of multiple physical apparatus in respective real-world locations,

establishing a respective presence for the physical apparatus in a respective one of the zones based on mappings between the respective real-world location and the respective zone, and

creating a respective object that represents the physical apparatus in the respective zone and is associated with a respective interface for communicant interaction with the physical apparatus;

transmitting to each of one or more of the respective network nodes a respective specification of a visualization of a spatial layout of the zones, graphical representations of the objects in their respective zones of presence, and graphical representations of avatars representing communicants in their respective zones of presence;

provisioning respective data stream connections between respective ones of the physical apparatus and respective ones of the network nodes.

39. The method ofclaim 38, wherein for each of one or more of the physical apparatus, the establishing of the presence of the physical apparatus is performed responsive to receipt of a login request that identifies the particular physical apparatus and is generated by the particular physical apparatus.

40. The method ofclaim 38, further comprising for each of one or more of the physical apparatus, in the respective zone of presence of the physical apparatus publishing one or more physical space data streams comprising respective representations of human perceptible physical stimuli in the respective real-world location, wherein the provisioning comprises provisioning data stream connections for transmitting respective ones of the published physical space data streams to respective ones of the network nodes for transforming into human perceptible stimuli.

41. The method ofclaim 40, wherein:

each of one or more of the network nodes publishes one or more respective client data streams; and

the provisioning comprises provisioning data stream connections for transmitting respective ones of the published client data streams to respective ones of the client network nodes for transforming into human perceptible stimuli, and transmitting respective ones of the published client data streams to respective ones of the physical apparatus for transforming into human perceptible stimuli in the respective real-world locations.

42. The method ofclaim 38, wherein a particular one of the physical apparatus is operable to perform a respective function in its respective real-world location in response to data transmitted by a particular one of the network nodes on a respective one of the data stream connections provisioned based on a request from the particular network node referencing the object representing the particular physical apparatus.

43. The method ofclaim 42, further comprising based on a notification of an event relating to the respective function performable by the particular physical apparatus, sending a notification of the event to the particular one of the network nodes of a respective one of the communicants.

44. The method ofclaim 43, wherein the particular physical apparatus comprises a printer, and the sending comprises sending to the particular network node a notification that a document has been printed.

45. The method ofclaim 43, wherein the particular physical apparatus comprises a facsimile machine, and the sending comprises sending to the particular network node a notification of an incoming facsimile.

46. The method ofclaim 43, wherein the particular physical apparatus comprises a telephony device, and the sending comprises sending to the particular network node a notification of an incoming telephone call.

47. The method ofclaim 42, wherein the particular physical apparatus comprises a printer, and further comprising based on a request from the particular network node referencing the object representing the particular physical apparatus, provisioning at least one data stream connection for the particular network node to print a document.

48. The method ofclaim 42, wherein the particular physical apparatus comprises a facsimile machine, and further comprising based on a request from the particular network node referencing the object representing the particular physical apparatus, provisioning at least one data stream connection for the particular network node to one of send a facsimile and receive a facsimile.

49. The method ofclaim 42, wherein the particular physical apparatus comprises a telephony device, and further comprising based on a request from the particular network node referencing the object representing the particular physical apparatus, provisioning at least one data stream connection for the particular network node to one of place an outgoing telephone call and receive an incoming telephone call.

50. A method, comprising:

administering a virtual area in a virtual communications environment, wherein the virtual area comprises one or more zones each of which defines a respective persistent context for realtime communications between network nodes of respective communicants who are present in the zone, and the administering comprises for each zone administering realtime communications between the respective network nodes of communicants who are co-present in the zone;

for each of respective ones of the communicants who are present in the virtual area, transmitting a respective specification of a visualization of the virtual area comprising graphical representations of the one or more zones and avatars respectively representing the communicants in the one or more zones in which they respectively have presence;

from sensing apparatus co-located with a particular one of the communicants in a physical space, receiving state information describing information relating to a physical state of the particular communicant;

based on the state information, updating the specification of the visualization of the virtual area and the avatars and transmitting the updated specification to each of respective ones of the communicants who are present in the virtual area.

51. The method ofclaim 50, wherein the state information describes the current real-world location of the particular communication, and the updating comprises providing an indication of the current real-world location of the particular communicant in the visualization.

52. The method ofclaim 51, wherein the updating comprises locating the avatar representing the particular communicant in a zone of the virtual area associated with the current real-world location of the particular communicant.

53. The method ofclaim 51, wherein the updating comprises providing a descriptive label indicative of the current real-world location of the particular communicant in association with the graphical representation of the avatar representing the particular communicant.

54. The method ofclaim 50, wherein the state information describes the state of the particular communicant in relation to a physical device associated with the particular communicant, and the updating comprises updating the graphical representation of the avatar representing the particular communicant based on the state of the particular communicant in relation to the physical device.

55. The method ofclaim 53, wherein the physical device is a headset; and based on a determination that the state information indicates that the headset is being worn by the particular communicant, the updating comprises including a graphical representation of a headset with the graphical representation of the avatar of the particular communicant.

56. The method ofclaim 50, wherein the state information describes a physical relationship between the particular communicant and another one of the communicants present in the virtual area, and the updating comprises updating the graphical representations of the avatars of the particular communicant and the other communicant to reflect the physical relationship between the particular communicant and the other communicant.

57. The method ofclaim 56, wherein based on a determination that the state information indicates that the particular communicant and the other communicant are co-located in a shared real-world location, the updating comprises including with the graphical representations of the avatars of the particular communicant and the other communicant respective indications that the particular communicant and the other communicant are physically co-located.

Images