US20050068905A1 - Audio/video-conferencing using content based messaging - Google Patents

Abstract

A content based messaging audio and video conferencing system provides a messaging infrastructure for multimedia and collaborative applications. The audio/video conferencing system may be implemented using a publish/subscribe infrastructure, such as a content-based messaging (CBM) infrastructure and system. Conference participants may exchange audio and/or video data streams using the CBM system. A list of contacts displays the current status of users. For example, the status of a user may be offline, online, and in a conference. The conference system uses the publish/subscribe infrastructure to provide instant state changes in the contact list using published updates to generate state messages for those users subscribing to contacts using the conference system.

Description

TECHNICAL FIELD
• The following description relates to network communications, and in particular to audio/video conferencing.
BACKGROUND
• Developments in communication technology allows individuals to inexpensively communicate over long distances. In particular, conferencing systems allow groups of individuals to have real-time audio and/or video conferences despite being in different locations.
• Conventional audio/video conference systems allow multiple users to connect into a central server to join a conference that is conducted by a host. However, such systems require that the participants connect to the server and that the host controls or administers the conference. If the host departs, the conference must end, or another participant must take the place of the departed host. Often, this is inconvenient in real world scenarios. However, new methods and technology continue to develop to supplement traditional audio/video conference systems.
SUMMARY
• In one general aspect, a content based messaging (CBM) audio and video conferencing system provides a messaging infrastructure for multimedia and collaborative applications. The audio/video conferencing system may be implemented using a publish/subscribe infrastructure, such as a CBM infrastructure and system. Conference participants may exchange audio and/or video data streams using the CBM system. A user may initiate a conference by inviting other users to join a conference or a user may join a conference that is already in progress.
• In another general aspect, a conference may be conducted by subscribing to a conference data stream of a content based messaging (CBM) network. Data conference messages are published and messages including the conference data stream corresponding to the subscription are received from the CBM network. The messages are presented to a user. The messages may include video data and/or audio data.
• The received messages may be assembled into streaming video and/or audio data and presented to the user. The messages to the user may be converted from a real time transport protocol to a format required by the Java Media Framework (JMF) to present the streaming audio and/or video data to the user.
• A conference may be initiated by inviting a participant connected to the CBM network to join the conference. A conference also may be initiated by inviting one or more participants connected to the CBM network to join the conference and subscribing to a conference data stream of a content based messaging CBM network associated with each participant. Initiating a conference by inviting one or more participants connected to the CBM network to join the conference may include selecting the one or more participants from a contact list of identifiers associated with each participant.
• Participants may enter or leave a conference at any time. For example, users may participate in a conference that is already established. The conference is not hosted or tied to a specific participant. As a result, all conference participants are equal.
• In addition, no central conference server is needed. Data may be routed through the messaging infrastructure, and audio and video data streams are replicated by the messaging infrastructure. As a result, no redundant transmission of data streams from a source is required. Conference participants may subscribe to messages that are important to them, for example, messages containing multimedia data (e.g., audio/video data) pertaining to a specific conference. Users also may subscribe to status changes of contacts in their contact list.
• According to another general aspect, a content based messaging network for conducting a conference between two or more participants includes an interface to receive subscriptions to a conference data stream corresponding to the conference and to receive publishing conference data messages corresponding to the conference, a processor to compute the subscriptions, to generate messages including the conference data stream, and to send the messages to client devices corresponding to the computed subscriptions; and an output to deliver the messages. The messages may include one video data, audio data, or a combination of video and audio data.
• The system also may include two or more client devices configured to generate the subscriptions and to receive the conference messages and present the messages to an associated user. Each client device is configured to assemble the received messages into streaming audio data, video data, or a combination of the two and be presented to the user. Each client device also may be configured to convert the received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user. Each client device is configured to receive an invitation to join the conference and to subscribe to a conference data stream of a content based messaging (CBM) network associated with each conference participant.
• The interface may be configured to receive subscriptions to a status of each conference participant and the processor is configured to receive published status information and to generate status messages based on the received subscriptions.
• According to another general aspect, a client device for conducting a conference comprises an interface to receive messages including a conference data stream from a CBM network; a processor to subscribe to the conference data stream received from the content based messaging (CBM) and to process the messages for presentation to a user, and a user interface to present the processed messages. The user interface is configured to present messages including video data, audio data, and/or a combination of video and audio data.
• The processor also may be configured to assemble the received messages into streaming video data, streaming audio data, or a combination of the streaming audio and video data for presentation to the user. The processor also may be configured to convert received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user. The processor is configured to publish messages associated with the conference to the CBM network.
• In another general aspect, a computer program product comprises instructions to cause a processor to subscribe to a conference data stream of a content based messaging (CBM) network; publish conference data messages; and process the conference data stream corresponding to the subscription from the CBM network for presentation to a user. The computer program product also may include instructions to process video data and audio data. The computer program product also may include instructions to assemble the received messages into streaming video data and/or streaming audio data for presentation to the user, and/or to publish messages associated with the conference to the CBM network. The computer program product also may include instructions to convert received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user. The computer program product also may include instructions to initiate a conference by inviting one or more participants connected to the CBM network to join the conference and/or to subscribe to a conference data stream of a content based messaging CBM network associated with each participant.
DESCRIPTION OF DRAWINGS
• FIG. 1 is a block diagram of an exemplary audio/video conference system.
• FIG. 2 is an exemplary architecture for the conference system ofFIG. 1.
• FIG. 3 illustrates the exemplary flow of data in the architecture ofFIG. 2.
• FIG. 4 is an exemplary graphical user interface for use with the conference system ofFIG. 1.
• FIGS. 5, 6,7,8, and9 are exemplary graphical user interfaces for establishing a conference.
• FIGS. 10, 11,12, and13 are exemplary graphical user interfaces for selecting options for a conference.
• Like reference symbols in the various drawings indicate like elements.
DETAILED DESCRIPTION
• System Overview
• The following description details an audio/video conference system that provides a content based messaging CBM infrastructure for multimedia and collaborative applications. The conference system may be implemented using a publish/subscribe infrastructure that includes at least two main subsystems: a multi-user audio/video conference system and a user management system that indicates the status (e.g., offline, online, or currently in a conference) of conference system users. Each subsystem is described in further detail below.
• Referring toFIG. 1, aconference system100 may include one ormore client devices110 and aCBM network120. Theclient devices110 may exchangedata streams130 and135 with theCBM network120 usinginformation paths140 to conduct audio and/or video conferences.
• Theclient devices110 may be operated by one or more users to access theCBM network120. An example of aclient device110 is a general-purpose computer capable of responding to and executing instructions in a defined manner.Client devices110 also may include a special-purpose computer, a personal computer (“PC”), a workstation, a server, a laptop, a Web-enabled phone, a Web-enabled personal digital assistant (“PDA”), an interactive television set, a set top box, an on-board (i.e., vehicle-mounted) computer, or a combination of two or more these devices capable of responding to, generating, and/or executing instructions. Theclient device110 may include any number of other devices, components, and/or peripherals, such as memory/storage devices, input devices, output devices, user interfaces, and/or communications interfaces.
• Theclient device110 also may include one or more software applications (e.g., an operating system, a browser application, a microbrowser application, a server application, a proxy application, a gateway application, a tunneling application, and a distributed conferencing application, among others) loaded on theclient device110 to command and direct theclient device110. Applications may include a computer program, a piece of code, an instruction, or some combination thereof, for independently or collectively instructing theclient device110 to interact and operate as desired.
• The applications may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, storage medium, or propagated signal or wave capable of providing instructions to theclient110 device. In particular, the applications may be stored on a storage medium or device (e.g., a read only memory (ROM), a random access memory (RAM), a volatile/non-volatile memory, a magnetic disk, or a propagated signal) readable by theclient device110, such that if the storage medium or device is read by theclient device110, the specified step or instructions are performed.
• Theclient device110 also may include one or more communications interfaces that allow the client device to send and receive information using thecommunications paths140 associated with theconferencing system100.
• Thecommunications paths140 may be configured to send and receive signals (e.g., electrical, electromagnetic, or optical) that convey or carry data streams (e.g.,130 and135) representing various types of analog and/or digital content. For example, thecommunications paths140 may be implemented using various communications media and one or more networks comprising one or more network devices (e.g., servers, routers, switches, hubs, repeaters, and storage devices). The one or more networks may include a WAN, a LAN, a broadband network, a plain old telephone service (POTS) network, a digital user line (DSL) network, an integrated services digital network (ISDN), and a synchronous optical network (SONNET), or a combination of two or more of these networks. In addition, thecommunications paths140 may include one or more wireless links that transmit and receive electromagnetic signals, such as, for example, radio, infrared, and microwave signals to convey information.
• TheCBM conference system100 uses a publish/subscribe technique to exchange data streams between theclient devices110. According to this technique, eachclient device110 may be an information consumer or producer. As an information producer, theclient device110 uses the conferencing application to publish information (e.g., an audio/video data of a conference) to theCBM network120. As an information consumer, theclient device110 uses the CBM conference application to register a “subscription” with theCBM network120 to specific published information (e.g., an audio/video stream of a conference). TheCBM network120 provides notifications (i.e., messages) including the published information to thoseconsumer client devices110 who subscribe to the information.
• TheCBM conference system100 may be implemented using a distributed conference application that includes a number of CBM conference applications running on each of theclient devices110. The CBM conference applications, in conjunction with aCBM network120, provides CBM conferencing and CBM status services that may be provided to each of theclient devices110. The CBM conference applications encode the audio and/or video data streams and publish corresponding conference information including the audio and/or video data to theCBM network120. The CBM applications also register subscriptions to the published conference data. The CBM conference applications receive messages corresponding to the subscriptions that include the conference data and process the received messages into a streaming data format for presentation to a user.
• TheCBM network120 may include one or more data processing and distribution devices (e.g., a CBM server, associated communications media, and data transport systems). For example, theCBM network120 may include one or more filtering CBM servers and communications interfaces (configured to communicate with communications paths140) that receive published information (e.g., pertaining to a conference or the status of system users) and generate messages that are transmitted to the consumers who subscribe to the published information. The filtering server may compute the registered subscriptions of theclient devices110 that match the published information and send the messages including the data that are sent to the subscribing consumers based on the computed subscriptions. In particular, the published information and messages include audio and/or video content in addition to user status data, as explained in detail below.
• Referring toFIG. 2, an exemplaryCBM conference architecture200 that is similar to an open systems interconnection (ISO) architecture may be used to implement theCBM conference system100. At the highest layer (e.g., the application layer), the distributed CBM conference application provides a user interface at auser interface layer210. Theuser interface layer210 allows a user to participate in an audio/video conference and determine the status of other users of theCBM conference system100. Theuser interface layer210 presents multimedia data streams received as messages from theCBM network120. The user interface manages multimedia data provided from the data capture devices (e.g., a video camera and/or a microphone). Theuser interface layer210 also generates and processes multimedia data streams that are published to theCBM network120 with the aid of theintermediate layer220 and theCBM adaptation layer230. The user interface also provides the user with various conference management features (e.g., creating a conference, negotiating audio and/or video capabilities, joining/leaving a conference, and closing a conference), as explained below.
• Theintermediate layer220 provides an interface between theuser interface layer210 and theCBM adaptation layer230. Theintermediate layer220 may be implemented using a Java Media Framework (JMF). The JMF is an application program interface (API) that enables audio, video, and other time based media to be added to java applications and applets. The JMF uses the Real-Time Transport Protocol (RTP) (e.g., RFC 3267) for communication between distributed components designed for multimedia communications. The JMF provides an interface between the data protocols of the user interface (and its associated data capture devices) and the CBM infrastructure. The JMF converts the multimedia data stream generated by the user interface layer to data packets in JAVA using the RTP protocol for publishing to theCBM network120. The JFM also converts RTP protocol packets received as messages form theCBM adaptation layer230 into a multimedia data stream that is compatible with theuser interface layer210 and its data capture devices. The JMF provides both “reliable” and “unreliable” communication modes. The reliable mode is used for all multimedia stream management functions. The unreliable mode is used for transmitting the multimedia data packets to theCBM network120.
• TheCBM adaptation layer230 is responsible for mapping of the RTP encoded multimedia packets onto messages used for communication inCBM network120. Theadaptation layer230 may be implemented in JAVA. Theadaptation layer230 also manages the necessary subscriptions for receiving published messages that include the multimedia data streams. Theadaptation layer230 is responsible for the mapping of java programming language calls onto CBM Messages. The interface that the adaptation layer implements is operable to abstract from specific network infrastructures (messaging, http, TCP/IP), so as to simplify mapping onto these network infrastructures. This abstract networking interface is defined by the JMF. Theadaptation layer230 is implemented in Java. Theadaptation layer230 takes information that is provided through the user interface by the JMF and maps this information onto CBM messages. No further encoding or decoding is necessary. The messaging infrastructure can handle messages of (basically) arbitrary length. There is a one-to-one mapping between packets from the JMF and the messages. Theadaptation layer230 also manages the connection of theclient device110 to theCBM network120 and the necessary subscriptions to receive the appropriate messages corresponding to a conference.
• The status of contacts andconference management layer250 provides an interface between theuser interface210 and theCBM network120 for communication and may be implemented using JAVA. The status of contacts andconference management layer250 provides all ftnctionality to publish and receive the presence information of contacts and conference participants. The status of contacts andconference management layer250 layer is responsible for receiving presence updates of all configured contacts and to publish information about changes of the state of its associated user. The status of contacts andconference management layer250 also manages the necessary subscriptions to receive the appropriate information and creates the messages to send state updates to other users. Thelayer250 also handles the connection to theCBM network120.
• TheCBM network120, as it is illustrated inFIG. 2, corresponds to the relevant physical layer for implementing theCBM conference system100. Thus, each distributed peer application in theCBM conference system100 communicates using theCBM network120 and its corresponding infrastructure. TheCBM network120 may be implemented using a publish/subscribe system, such as, for example, the Elvin messaging architecture developed by DSTC in Brisbane Australia. Although the examples described herein refer to Elvin, other CBM protocols and systems, such as, for example, CosNotif, JMS, Keryx, and Gryphon may be used to distribute information to theclient devices110 using their associated CBM conference applications.
• Referring toFIG. 3, the flow of data is shown with regard to the exemplary conference architecture ofFIG. 2. An input device (e.g., amicrophone301 and/or a video camera302) may be used to generate corresponding streams ofdata310 and315, respectively. Theuser interface layer210 manages the audio310 and/orvideo315 data streams by providing any necessary encoding or processing. The audio/video data streams are provided from the user interface to theJMF220, which encodes the data into packets using the RTP. The RTP encoded data packets are provided to theCBM adaptation layer230 for encoding as CBM messages, and are published to theCBM network120.
• Similarly, theCBM adaptation layer230 registers a subscription to specific conference data, and receives messages corresponding to the subscriptions for conference data (e.g., including audio/video data packets registered with theCBM network120. The messages are decoded from RTP by theJMF220 and provided to the user interface for presentation to the user using one or more data output devices325 (e.g., a display and/or speakers).
• Theuser interface layer210 also provides user status information to the status/conference management layer250 asdata stream330. For example, theuser interface layer210 provides the status of an associated user to the statusconference management layer250 indicating whether the user is offline, connected to theCBM network120 and able to engage in a conference, or is online and engaged in a conference. The statusconference management layer250 publishes the status to theCBM network120 where it is provided to other subscribing users that list the user as a contact in their contact list. The user status is published to the CBM network120 (where it is delivered subscribing to contact lists).
• The status/conference management layer250 also receives status data streams340 corresponding to contacts of the associated user interface's contact list to which it subscribes. Thestatus data stream340 is delivered to theuser interface layer210 and presented to the user in the contact list. In this way when the state of a user changes, the information is distributed to all users of theCBM conference system100 who list a user in their contact lists.
• As shown inFIG. 3, those parts of the architecture that consume messages (e.g., receive multimedia data, receive status updates) are independently responsible for creating the appropriate subscriptions with theCBM network120. The subscriptions ensure that only required messages are sent to the corresponding CBM conference applications. As a result, the required bandwidth and the processing power needed by theCBM conference system100 is reduced.
• Referring back toFIG. 1, an example of a conference with data streams between five peer conference applications is described. According to the example, peer applications A, B, and C are engaged in a conference. Each of the peers exchanges multimedia data that are published or received as messages by theCBM network120. The published messages are published once from eachclient device110 to theCBM network120. Each of the peer applications participating in the conference subscribe todata streams135 including the conference specific data messages, and each conference specific message is sent to each of the subscribing participants (e.g., peer applications A, B, and C).
• For example,multimedia messages145 published from A corresponding to the conference are sent to theCBM network120. Peers B and C subscribe to the messages in themultimedia data stream135. The published messages from A are matched by the CBM network to B and C's conference subscriptions and are sent to both B and C. Similarly, amessage150 and amessage155 corresponding to multimedia streams published by B and C, respectively, are matched to a conference subscription by A and are delivered to A by the CBM network.
• In addition, according to this example, each of the users associated with the peers A, B, C, and D is listed in the contact lists of each of the associated user interfaces. Therefore, the peers A-D publish and subscribe tostatus messages160 from each of the peers to maintain the status of the each user in their associated contact list. For example, the status messages exchanged indicate that A, B, and C are in a conference and D is online (but not in a conference). In this case, D only subscribes to status messages of the users listed in its contact list (but not to any multimedia streams).
• As part of this example, peer E is not in a conference and does not belong to the contacts of A, B, C, or D. As a result, peer E does not subscribe to or publish any messages.
• In one implementation, a quenching feature (provided in Elvin) may be used to provide feedback information to a producer about consumers that subscribe to data published by the producer. In theCBM conference system100, quenching may be used to determine information about peers in a conference (e.g., if any peers remain in a conference). If the quenching mechanism of a producer of a multimedia stream is activated and determines that no subscribing peers remain in a conference, the producer stops publishing the data stream as messages to theCBM network120. The quenching mechanism ensures that whenever a new peer is added to the conference, any producers are informed to start sending the multimedia data stream associated with the conference. Quenching may be configured using the options interface, as described below.
• Graphical User Interfaces
• FIG. 4 shows an exemplary conference systemgraphical user interface400. Thegraphical user interface400 may include amenu bar401, avirtual button bar410, acontact list window420, aconference area440, and astatus bar450.
• A user may use themenu bar401 to access dropdown menus. The dropdown menus may be used to select CBM conference options, CBM conference user management features, and to set up and control the CBM conference user interface and its features.
• Thevirtual button bar410 may include for example four virtual buttons:button411 to create a CBM conference with a contact selected from thecontacts list422, button413 to join a CBM conference already in progress,button415 to leave a conference, andbutton417 to display a preview window. The virtual buttons may be selected using an input device (e.g. a mouse) of theclient device110 to invoke their associated functions.
• Thecontact list window420 includes a list of user contacts422 (e.g., users of theconference system100 with whom a user communicates or monitors their status). Thecontact list422 includes one or more identifiers (e.g., screen names or e-mail addresses) associated with users of theconference system100. The current status of the contacts included in thecontact list422 may be determined by viewing a visual identification associated with a corresponding user identifier.
• In one implementation, the status is indicated by the color of a bubble contact identifier associated with each identifier. For example, red may be used to indicate that a contact is offline, green may be used to indicate that a contact is online and not in a conference, and blue may be used to indicate that a contact is online and in a conference.
• Thecontact list window420 may use folders to organize groups of contacts. For example, the folders may group users by their current status, and as shown include a folder for online contacts and a folder for offline contacts.
• Theconference area440 may be used to present information about ongoing conferences including data streams corresponding to multimedia data streams of each conference participant. Theconference area440 may include one or more conference windows (e.g.,460) that present the audio/video representations corresponding to the conference participants. In addition, amonitor window465 may be displayed to preview and control the data stream sent toother client devices110 during a conference.
• Eachconference window460 may include atitle bar470 containing an identifier of a corresponding conference participant (e.g., dominic@bne). Theconference window465 also includes astatus bar475 at the bottom indicating the time an associated participant has been engaged in a conference. A small icon or other indication477 (e.g., a “film” symbol) may be selected to obtain detailed information (e.g., data-rates, frame-rates, encoding of the current data stream, and a current time-stamp) about the audio/video stream that is presented in the corresponding conference window.
• Themonitoring window465 may include a title bar467 (e.g., preview). Themonitoring window465 does not include a status bar because the data presented in the window is generated locally and not by the JMF. Themonitoring window465 may be used to control the data stream published by theclient device110 associated with themonitoring window465 that is sent to the conference participants.
• Thestatus bar450 indicates information about theconference system100. For example, thestatus bar450 may include an indication that theclient device110 corresponding to theuser interface400 is connected to theCBM network120. Thestatus bar450 also may indicate information regarding memory consumption of thecorresponding client device110.
• Creating a Conference
• To create a conference, a user may select a contact from theircontact list422. The user then selects the “create conference”virtual button411 in thevirtual button bar410 to start the conference process.
• In response, thegraphical interface500 as shown inFIG. 5 may be launched and used to configure the user interface/conference settings. If a change is desired, achange setting input510 may be selected and avirtual button520 may be selected to continue the process. If not, a conference is set up using the settings used in a previous conference and the virtual button530 (e.g., Finish) may be selected.
• FIGS. 6 and 7 show exemplary graphical user interfaces that may be used to select options and to change the user interface settings (e.g., used by the JMF). As shown inFIG. 6, theinterface600 allows a user to select data capture devices associated with theclient device110 that may be used to conduct a CBM audio/video conference. A tab may be associated with each data capture device. For example, avideo device tab610 and anaudio device tab620 are provided to select and set up settings for each corresponding audio and video data capture device.
• Avideo device tab610 may include aselection input box630 to activate the video device for which settings have been selected (or default settings provided by the CBM system100). A type of video device may be selected from a drop downmenu638. Selections for encoding, video size, frame rate, and bits per pixel also may be selected using drop down menus. For example, a frame rate between 5 and 10 Frames/sec may be selected using the interface. Using thevideo tab620, the video size may be reduced to provide better video quality. A video encoding scheme, such as Video for Windows (i.e., “vfw: . . . ”) may be selected. Virtual buttons to finish or proceed to other screens also may be provided.
• Similarly, as shown inFIG. 7, theAudio device tab620 may be used to select audio device settings. Aselection box720 may be used to enable the audio device.Dropdown menus730,740, and750 may be used to select a driver (e.g., Java Sound audio capture), encoding, and sample rate, respectively.Radio buttons760 for bits per sample (e.g., 8 bit or 16 bit), channels (e.g., mono or stereo), endian (e.g., big or little) may be selected.
• After clicking theNext button770, ascreen800, as shown inFIGS. 8 and 9, for specifying the output parameters appears. A drop downmenu810 may be used to select a format (e.g., RAW/RTP).Tabs820 and830 specify output parameters for audio and video, respectively. For example, drop downmenus840 and841 may be used to specify encoding and sample rate.Radio buttons850 for bits per sample (e.g., 8 bit or 16 bit), channels (e.g., mono or stereo), endian (e.g., big or little) also may be selected.
• As shown inFIG. 9, output parameters for video data associated withtab830 may be selected. Aselect box935 may be used to enable a track.Dropdown menus940,941,945, and950 may be used to select encoding, video size, and frame rate, respectively. AFinish button960 may be used to finalize the parameters and setup process.
• After this process is finished for the initiator of the conference, an invitation to the conference is sent to an invited user (previously selected using the contact list as described above). When connecting to theCBM network120, theCBM adaptation layer230 registers a subscription to an identifier associated with its user interface. This process allows any published invitation to be sent by theCBM network120 to be sent to the subscribing user interface. The invited user can then accept or reject the invitation. If the invitation is accepted, the invited user may proceed through the same process of configuring the data streams as described above for the conference initiator, and register a subscription for the conference messages. After all configurations have been selected, the CBM conference may begin.
• Installation
• Before a conference may be set up, theclient devices110 must install the necessary software and drivers (e.g., the camera and associated drivers, a microphone and associated drivers, Java (e.g., 1.3 or 1.4) and JMF (e.g., JMF 2.1)). After installing the JMF, the user may run the JMF registry (a tool provided by the JMF to register the audio and video capture devices associated with the user interface) from a programs menu of the operating system. The JMF also may be used to register and detect any data capture devices and to make the data capture devices available to the JMF and any applications using the JMF. Registration is necessary after installation of the JMF and after making any changes to the audio and video configurations.
• To run a CBM conference, theclient device110 also must access to theCBM network120 and register any subscriptions (conference data or user status data). TheCBM network120 includes one or more CBM servers and several client devices110 (e.g., that may be producers or consumers). For aclient device110 to gain access to the CBM network120 a client library is used. The client library is programming language and platform dependent. In this implementation, a client library is provided by Elvin. The client library provides the runtime support for applications using Elvin. The client library encapsulates the wire-level protocol understood by the server using classes or functions in a style appropriate to the specific language (in this case Java).
• FIGS. 10, 11,12, and13 show an optionsgraphical user interface1000 that may be used to configure options of the conference application. Theinterface1000 may include several tabs to configure the conference application. For example, ageneral tab1010, aconnection tab1020, asilence detection tab1030, and aconference options tab1040 may be provided. Virtual buttons save1045 and cancel1047 may be used to save or cancel option settings, respectively.
• Thegeneral tab1010 may include a field to insert a user identifier (e.g., an e-mail address, a screen name, or another system identifier). The identifier is used to identify a user on theCBM network120 and also is used in the contact list of other users to identify the user (e.g., to initiate a conference or to determine a user status). Aselection box1042 is also provided to enable the quenching feature.
• Referring toFIG. 11 theconnection tab1020 may be used to specify a CBM network server. In one implementation, the URL of the server may be selected from adropdown menu1050.Radio button selections1054 may be used to indicate whether a server is selected from the dropdown menu, if the server is entered manually (e.g., a desired server's URL is typed in), or if the server is to be automatically detected. Once the server is selected, avirtual button1057 may be selected to check the connection of theclient device110 to the selected conference server. The user is then informed about successful and unsuccessful attempts to connect in the status bar of the main window.
• As shown inFIG. 12 thesilence detection tab1030 may be used to select silence detection. Silence detection is used to reduce the required bandwidth. For example, data may be transmitted only when the user is talking. If silence detection is enabled, no data is sent to the other conference participants if the sound level is below a given threshold. Aselection box1060 may be use to enable the silence detection. Slidingscales1062 and1064 may be used to establish the minimum threshold and number of silent buffers respectively. The silent buffers specify the time of continuing silence (input gain below the threshold) before the system stops sending data. This means that after the input gain is below the threshold for some time (e.g., in milliseconds), transmitting of data is stopped. In addition, ascreen area1066 is provided to display activity detection of a user.
• InFIG. 13conference options tab1040 may be used to select the initial timeout that the conference interface waits for participants to reply to an invitation. Aselection box1070 may be used to select an open conference, in which any user of the CBM network may participate in the conference and/or the administration of the conference. Aselection scale1075 may be used to specify a conference time.
• A number of exemplary implementations have been described. Nevertheless, it is understood that various modifications may be made. For example, suitable results may be achieved if the steps of the disclosed techniques are performed in a different order and/or if components in a disclosed architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components. Accordingly, other implementations are within the scope of the following claims.

Claims (40)

1. A method of conducting a conference comprising:

subscribing to a conference data stream of a content based messaging (CBM) network;

publishing conference data messages;

receiving messages including the conference data stream corresponding to the subscription from the CBM network; and

presenting the messages to a user.

2. The method ofclaim 1 wherein presenting the messages to the user includes presenting video data to the user.

3. The method ofclaim 1 wherein presenting the messages to the user includes presenting audio data to the user.

4. The method ofclaim 1 wherein presenting the messages to the user includes presenting audio and video data to the user.

5. The method ofclaim 1 wherein presenting the messages to the user further comprises assembling the received messages into streaming video data and presenting the streaming video data to the user.

6. The method ofclaim 1 wherein presenting the messages to the user further comprises assembling the received messages into streaming audio data and presenting the streaming audio data to the user.

7. The method ofclaim 1 wherein presenting messages to the user further comprises assembling the received messages into streaming audio and video data and presenting the streaming audio and video data to the user.

8. The method ofclaim 1 wherein presenting the messages to the user further comprises converting received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user.

9. The method ofclaim 1 further comprising initiating a conference by inviting a participant connected to the CBM network to join the conference.

10. The method ofclaim 1 further comprising initiating a conference by inviting one or more participants connected to the CBM network to join the conference and subscribing to a conference data stream of a content based messaging CBM network associated with each participant.

11. The method ofclaim 10 wherein initiating a conference by inviting one or more participants connected to the CBM network to join the conference further comprises selecting the one or more participants from a contact list of identifiers associated with each participant.

12. The method ofclaim 11 further comprising subscribing to a status of each conference participant and presenting the status to the user.

13. The method ofclaim 11 further comprising publishing a status of the user to the CBM network.

14. A content based messaging network for conducting a conference between two or more participants comprising:

an interface to receive subscriptions to a conference data stream corresponding to the conference and to receive publishing conference data messages corresponding to the conference;

a processor to compute the subscriptions, to generate messages including the conference data stream, and to send the messages to client devices corresponding to the computed subscriptions; and

an output to deliver the messages.

15. The system ofclaim 14 wherein the messages include one of video data, audio data, or a combination of video and audio data.

16. The system ofclaim 14 further comprising two or more client devices configured to generate the subscriptions and to receive the conference messages and present the messages to an associated user.

17. The system ofclaim 16 wherein each client device is configured to assemble the received messages into streaming video data and to present the streaming video data to the user.

18. The system ofclaim 16 wherein each client device is configured to assemble the received messages into streaming audio data and to present the streaming audio data to the user.

19. The system ofclaim 16 wherein each client device is configured to assemble the received messages into streaming audio and video data and to present the streaming audio and video data to the user.

20. The system ofclaim 16 wherein each client device is configured to convert the received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and video data to the user.

21. The system ofclaim 16 wherein each client device is configured to receive an invitation to join the conference and to subscribe to a conference data stream of a content based messaging (CBM) network associated with each conference participant.

22. The system ofclaim 16 wherein the interface is configured to receive subscriptions to a status of each conference participant and the processor is configured to receive published status information and to generate status messages based on the received subscriptions.

23. A device for conducting a conference comprising:

an interface to receive messages including a conference data stream from a CBM network;

a processor to subscribe to the conference data stream received from the content based messaging (CBM) and to process the messages for presentation to a user; and

a user interface to present the processed messages.

22. The device ofclaim 23 wherein the user interface is configured to present messages including video data.

24. The device ofclaim 23 wherein the user interface is configured to present messages including audio data to the user.

25. The device ofclaim 23 wherein the user interface is configured to present messages including audio and video data to the user.

26. The device ofclaim 23 wherein the processor is configured to assemble the received messages into streaming video data for presentation to the user.

27. The device ofclaim 23 wherein the processor is configured to assemble the received messages into streaming audio data and the user interface is configured to present the streaming audio data to the user.

28. The device ofclaim 23 wherein the processor is configured to assemble the received messages into streaming audio and video data and the user interface is configured to present the streaming audio and video data to the user.

29. The device ofclaim 23 wherein the processor is configured to convert received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user.

30. The device ofclaim 23 wherein the processor is configured to publish messages associated with the conference to the CBM network.

31. A computer program product comprising instructions to cause a processor to:

subscribe to a conference data stream of a content based messaging (CBM) network;

publish conference data messages; and

process the conference data stream corresponding to the subscription from the CBM network for presentation to a user.

32. The computer program product ofclaim 31 wherein the instructions to process the messages include instructions to process video or audio data.

33. The computer program product ofclaim 31 further comprising instructions to assemble the received messages into streaming video data for presentation to the user.

34. The computer program product ofclaim 31 further comprising instructions to assemble the received messages into streaming audio data for presentation to the user.

35. The computer program product ofclaim 31 further comprising instructions to assemble the received messages into streaming audio and video data for presentation to the user.

36. The computer program product ofclaim 31 further comprising instructions to convert received messages from a real time transport protocol to a format required by the Java Media Framework (JMF) to present streaming audio and/or video data to the user.

37. The computer program product ofclaim 31 further comprising instructions to initiate a conference by inviting a participant connected to the CBM network to join the conference.

38. The computer program product ofclaim 37 further comprising instructions to subscribe to a conference data stream of a content based messaging CBM network associated with each participant.

39. The computer program product ofclaim 31 further comprising instructions to publish messages associated with the conference to the CBM network.

Images