This is a continuation application of U.S. Reissue Application Ser. No. 10/020,515, filed Dec. 7, 2001, which is a reissue application of U.S. Pat. No. 5,999,977, issued Dec. 7, 1999, which is a continuation of U.S. patent application Ser. No. 08/468,715, now abandoned, filed Jun. 5, 1995, which is a continuation of U.S. patent application Ser. No. 08/396,198, filed Feb. 24, 1995, now U.S. Pat. No. 5,854,898 issued Dec. 29, 1998. Notice: More than one reissue application has been filed for the reissue of U.S. Pat. No. 5,999,977. The reissue applications are application Ser. Nos. 10/020,515, 10/857,798, 10/857,799, 10/857,805, and 10/857,806 (the present application).
BACKGROUND OF THE INVENTION1. Field of the Invention
The present invention relates to teleconferencing systems.
More specifically, the present invention relates to the addition of a data stream, such as an auxiliary data stream to a teleconference.
2. Background Information
Teleconferencing is increasingly becoming a popular application in personal computer systems. Such applications typically allow the transfer of audio and video data between users so that they can speak and otherwise communicate with one another. Such applications sometimes also include data sharing wherein various types of data such as documents, spreadsheets, graphic data, or other types of data, can be shared and manipulated by all participants in the teleconference. Different teleconference applications perhaps residing on different hardware platforms have different capabilities. Moreover, a wide variety of features has been implemented in different teleconference applications, and the proliferation of different types of computer systems with different capacities, and different networking media has created challenges for teleconferencing.
For example, for most teleconferencing applications, it is assumed that the sender and the receiver have certain minimum capabilities. However, with the wide diversity of systems having different computation capacities, and in addition, the wide variety of networking media, that certain systems may not have certain capabilities. For example, the first system may be a high performance workstation coupled to a high performance communication medium whereas a second system may employ an earlier generation processor, operate at a substantially slower clock rate, and/or be coupled to a lower capacity communication medium. Certain network capabilities such as multicast or other optimization features, may not be present in certain networking media. Thus, in order for some teleconference applications to function, the participants in the conference can only operate at the fastest possible configuration provided by any minimum system which may participate in the teleconference. Of course, this results in certain inefficiencies, especially if both of the participants are capable of transmitting in a higher capacity than the system with the least possible capability.
Another issue in teleconference applications is the ability of certain stations to participate in more than one teleconference. In fact, in certain circumstances, multiple individual conferences may be desired to be merged by a user according to operating circumstances. Due to the distributed nature of certain networks, during this merge operation, certain circumstances may change. That is, that while a single station is merging more than one conference it is participating in, a second station at a remote location may further have other operating circumstances changing (e.g., participants leaving, entering, or otherwise joining an on-going teleconference), and thus, the management of such merging operations becomes unduly burdensome.
Yet another shortcoming of certain prior art teleconference applications is the ability to associate an independent data stream with an on-going teleconference. For example, a source participant may desire to provide an additional data stream to other participants in a teleconference. This additional source may include, but not be limited to, video, data, audio or any other type of data available to the source participant. For example, such an additional source may include other audio information for a receiver. Other types of data may also be desired to be associated with an on-going teleconference, which may be accessible to other participant in the teleconference. Certain prior art teleconferencing applications lack these abilities.
SUMMARYA method and apparatus for optimizing transmission of data to a plurality of second endpoints in a system wherein a first endpoint is providing data to the plurality of second endpoints each connected by point-to-point communication channels. This may be useful in teleconferencing applications with a plurality of participants (endpoints) or broadcast server applications. The first endpoint activates a multicast communication channel having a first multicast address and commences broadcast of the data over the multicast communication channel. The first endpoint transmits a request message to each of the plurality of second endpoints in order to query each of the second endpoints whether they can receive transmissions broadcast to the first multicast address. Certain of the plurality of second endpoints transmit an acknowledgement message if they can receive transmissions broadcast to the first multicast address, and the first endpoint receives the acknowledgement message. Then, for each acknowledgement message received from certain of the plurality of second endpoints, the first endpoint deactivates the point-to-point communication channel and the certain of the plurality of second endpoints.
The broadcast of the data and the multicast communication channel is terminated if at least two of the plurality of second endpoints do not transmit the acknowledgement messages containing a positive acknowledgement. In this instance, communication channels are maintained as point-to-point. Subsequent to commencing broadcast of the data to the multicast address, the first endpoint can receive detach messages from certain of the plurality of second endpoints, and if at least two of the plurality of second endpoints are not receiving the data, then the first endpoint can terminate the broadcast of the data and the multicast communication channel. Communication of the data in this instance also reverts to point-to-point communication channels.
In implemented embodiments, the acknowledgement message includes a response code which indicates whether the second endpoint can receive transmissions broadcast to the first multicast address. Also, in implemented embodiments, prior to the first endpoint activating the multicast communication channel having the first multicast address, it is determined whether the single communication medium supports broadcasting to the first multicast address, and if not, multicast cannot be activated.
BRIEF DESCRIPTION OF THE DRAWINGSThe present invention is illustrated by way of example and not limitation in the figures of the accompanying in which like references indicate similar elements and in which:
FIG. 1 illustrates an example configuration in which various embodiments of the present invention may be implemented.
FIG. 2 shows a typical teleconferencing display which has both media and non-media sources displayed during the course of the teleconference.
FIG. 3 shows a single system in which embodiments of the present invention may be implemented.
FIG. 4 shows an example architecture of a system employing various embodiments of the present invention.
FIG. 5 shows a more detailed view of the conference component illustrated inFIG. 4.
FIG. 6 shows a sequence of typical conference events in a conference component which are issued to an application.
FIG. 7 shows a typical sequence of steps performed for member initialization within the conference component.
FIGS. 8-10 show typical exchanges of messages for different operations.
FIG. 11 shows a detail of a first endpoint establishing a teleconference.
FIG. 12 shows a sequence of steps performed in a second endpoint receiving the messages sent from a first endpoint during the establishment of a teleconference.
FIGS. 13-25 show details of the messages transmitted between endpoints during various teleconferencing applications.
FIGS. 26a,26b,26c, and26d show the steps taken for performing merge operations.
FIGS. 27a and 27b show the conferences before and after a merge operation between teleconferences.
FIGS. 28a-b show a sequence of steps performed by the conference component during a merge operation.
FIG. 29 shows an example of a merged conferences table within a single participant.
FIGS. 30a-30b shows a sequence of steps performed for converting point to point connections to multicast connections for a teleconference.
FIGS. 31 and 32 show the adding of an auxiliary source and the messages during the adding of the source to an existing teleconference.
FIGS. 33-34 show the details of a sequence of steps performed within a participant for adding an auxiliary source.
FIGS. 35a-35b show an example sequence of messages which are sent between a first endpoint and a plurality of other endpoints in a networking system, and showing various messages transmitted therebetween.
DETAILED DESCRIPTIONThe present invention relates to networking systems, more specifically, the present invention describes a messaging protocol for establishing and maintaining teleconference connections between a plurality of participants in a networking system. Applications for this include, transmitting application and/or system capabilities between participants or potential participants in a teleconference, notifying participants of a teleconference that more than one teleconference should be merged, and addition of an auxiliary data stream to an on-going teleconference. Although the present invention will be described with reference to certain specific embodiments thereof, especially, with relation to certain hardware configurations, data structures, packets, method steps, and other specific details, these should not be viewed as limiting the present invention. Various modifications and other may be made by one skilled in the art, without departing from the overall spirit and scope of the present invention.
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. Copyright Apple Computer, Inc.
A typical system configuration in which a teleconference may take place is illustrated as100 inFIG. 1. For example, afirst workstation150 may communicate via teleconference with asecond workstation155, as illustrated.System150 may include acentral processing unit150c which is coupled to adisplay150d avideo input device150a, and asound input device150b. Thesystem150 may communicate with oversystem155 overnetworking medium170 vianetwork connection module160.160 may include any number of network adapters commercially available such as using Ethernet, Token Ring, or any other networking standard commercially available. Note thatnetwork adapter160 may also include a wireless network adapter which allows transmission of data between components without a medium170. Communication is thus provided vianetwork adapter165 coupled tosystem155, and bi-directional communications may be established between two systems.System150 further has akeyboard150e and apointing device150f, such as a mouse, track ball, or other device for allowing user selections and user input.
A teleconference display is shown at200 atFIG. 2. In implemented embodiments of the present invention, there is a source window, such as201, showing a monitor of the local media source, and there are other media windows, such as202 or203 for each other user with which a participant is having communication. In the illustrated example, each of the windows201-203 provides media information, that is, real-time audio and/or video information for bi-directional teleconferencing. In alternate embodiments of the present invention, non-media information such as204 may also be displayed within the teleconferencing display. As will become apparent in the description below, in addition to media and non-media information, messaging information may also be transmitted between stations. In addition, an auxiliary media source (e.g. audio or video information) may be transmitted with a specified conference. The details of this will be discussed in more detail below.
In implemented embodiments of the present invention, a general purpose computer system is used for implementing the teleconferencing applications and associated processes to be described here. Although certain of the concepts to be described here will be discussed with reference to teleconferencing, it is apparent that the methods and associated apparatus can be implemented for other applications, such as file sharing, real time data acquisition, or other types of applications which sends data from a first participant to a second participant or set of participants. A computer system such as that used for implementing embodiments of the present invention will now be described.
A computer system, such as a workstation, personal computer orother processing apparatus150c or155c as shown inFIG. 1 is illustrated in more detail inFIG. 3. 150c comprises a bus or other communication means301 for communicating information, and a processing means302 coupled withbus301 for processing information.System150c further comprises a random access memory (RAM) or other volatile storage device304 (referred to as main memory), coupled tobus301 for storing information and instructions to be executed byprocessor302.Main memory304 also may be used for storing temporary variables or other intermediate information during execution of instructions byprocessor302. Included inmemory304 during run-time may be the conference component module which operates according to the communication protocols to be described below.System150c also comprises a read only memory (ROM) and/or otherstatic storage device306 coupled tobus301 for storing static information and instructions forprocessor302, and adata storage device307 such as a magnetic disk or optical disk and its corresponding disk drive.Data storage device307 is coupled tobus301 for storing information and instructions.
System150c may further be coupled to a displaydevice adapter display321 such as a cathode ray tube (CRT) or liquid crystal display (LCD) coupled tobus301 for displaying information to a computer user. Such adisplay321 may further be coupled tobus301 for the receipt of video or image information. Analphanumeric input device322, including alphanumeric and other keys may also be coupled tobus301 for communicating information and command selections toprocessor302. An additional user input device iscursor control323, such as a mouse, a trackball, stylus, or cursor direction keys, coupled tobus301 for communicating direction information and command selections toprocessor302, and for controlling cursor movement ondisplay321. For teleconferencing applications,system150c may also have coupled to it asound output device324, avideo input device325, andsound input device326, along with the associated D/A (Digital-to-Analog) and A/D (Analog-to-Digital) converters for inputting or outputting media signal bitstreams.System150c may further be coupled tocommunication device327 which is coupled tonetwork adapter160 for communicating with other teleconferencing stations.
Note, also, that any or all of the components ofsystem150c and associated hardware may be used in various embodiments, however, it can be appreciated that any configuration of the system may be used for various purposes according to the particular implementation.
In one embodiment,system150c is one of the Apple Computer® brand family of personal computers such as the Macintosh 8100 brand personal computer manufactured by Apple Computer, Inc. of Cupertino, Calif.Processor302 may be one of the PowerPC brand microprocessors manufactured by Motorola, Inc. of Schaumburg, Ill.
Note that the following discussion of various embodiments discussed herein will refer specifically to a series of routines which are generated in a high-level programming language (e.g., the C or C++ programming language) and compiled, linked, and then run as object code insystem150c during run-time withinmain memory304 byprocessor302. For example the object code may be generated by the C++ Compiler available from Symantec, Inc. of Cupertino, Calif.
Although a general purpose computer system has been described, it can be appreciated by one skilled in the art, however, that the following methods and apparatus may be implemented in special purpose hardware devices, such as discrete logic devices, large scale integrated circuits (LSI's), application-specific integrated circuits (ASIC's), or other specialized hardware. The description here has equal application to apparatus having similar function.
FIG. 4 illustrates a plurality of processes and/or apparatus which may be operative withinsystem150c. At the highest level, for example, at the highest level in the ISO/OSI networking model, anapplication program401, such as a teleconferencing application, an audio/video server, or a data server, communicates withconference component process400 in the form of Application Program Interface (API) calls. Theconference component400 allows the application to establish communications between two or more teleconference stations. Control information, and media information can be transmitted between the first participant system and a second participant system. The conference component will be shown in more detail inFIG. 5.Conference component400 communicates with thetransport component402 by sending MovieTalk messages for other teleconferencing stations which are encapsulated and placed into a form that thetransport component402, thenetwork component403, and thesystem network component404 can packetize and transmit overnetworking medium170. For the purposes of the remainder of this disclosure, certain of the MovieTalk API calls and MovieTalk messages which are transmitted between conference components in a teleconferencing system will be described in more detail.
Thetransport component402 and thenetworking component403 provide the necessary operations for communication over the particular type ofnetwork adapter160 andnetworking medium170 according to implementation. For example,networking component402 may provide the TCP or ADSP protcols and packetizing, according to those respective standards.
A more detailed view of theconference component400 is shown inFIG. 5. Specifically, theconference component400 is shown in twoportions400a and400b which show input and output portions of the conference component. Although illustrated as a separate transmitter and receiver, each conference component in the system has both capabilities, so that full bi-directional communication between conference components in respective participant teleconference systems in a network may communicate with one another. As illustrated, the input portion of theconference component400a receives video and sound information overmedia input channels510 and520. The video channel component andsound channel component504 present media data at regular intervals to sequencegrabber502. The real-time sound and video data (hereinafter referred to as “media data”) are provided to asource stream director500 fromsequence grabber502 which then provides the media messages to thetransport component402.Flow Control501 then lets the video and sound data flow through at an implementation-dependent frequency. Thevideo channel component503,sound channel component504, andsequence grabber502 all are implemented using prior art products such as those commercially available (e.g., the QuickTime video channel, sound channel components, and sequence grabbers, available from Apple Computer, Inc. of Cupertino, Calif.)Flow control501 may be implemented using known flow control apparatus and/or method as are commercially available, such as those which regulate flow based upon bandwidth, and other constraints in the source participant system. The conference component further comprises asink stream director510 which comprises a portion of thecomponent400b of the conference component for receipt of media data fromtransport component402. Correspondingflow control511, video andsound stream players512 and513, and compression andsound manager514 and515, for output ofvideo streams530 and540, also form part of the conference component for full bi-directional conferencing capabilities.
The conference component's main function is to establish and maintain a bi-directional connection between every member of a conference. Conferencing applications use a pre-established control channel to exchange control data that is pertinent to the conference. This data might include user identification information or other information that is germane to the application's operation. Conferencing applications (e.g.,401) define the format and content of these control messages by establishing their own control protocols. The conferencing component further establishes communication channels between a first endpoint and a second endpoint, using theunderlying transport component402. Thus, once a media channel has been established, the conference component uses thetransport component402's media channel which is provided for transmission of media and non-media information. For the remainder of this application, however, the focus will be on the establishment of communication between a first and second participant (referred to as endpoints) or group of participants which may participate in a teleconference.
Application Program Interface (API)Theapplication program401 controls theconference component400 by the issuance of MovieTalk application API calls. The conference component operates using an event-driven model wherein events pertinent to the application are issued to the application program. The application program can then take appropriate action either by modifying internal data structures within (creating a source or sink), and/or issuance of appropriate messages over the network to other connected participants, or potential participants. According to messages received by the conference component, a current context and API calls from the application, the conference component can take appropriate action.
A typical series of events which occur after the establishment of a teleconference by the conference component in an application is illustrated inFIG. 6 as600. For example, upon an initial desire by an application to enter into a conference (as expressed by an API call) or a call from another participant, a conference-ready event601 (e.g. mtConferenceReadyEvent) is generated. The application then creates a media source in the conference component (e.g., member A) which is to provide the conference information. Subsequent to that, any auxiliary media sources may also be attached to the main conference atstep610 for a second media source (e.g. by the call MTConferenceAttachAuxiliary Source). Then, any members that are new to the conference are recognized as being ready by the receipt of MemberReady (e.g. mtMemberReady) events (e.g.,602 and603 ofFIG. 6). This establishes the media sinks such as b and c illustrated inFIG. 6. Then, during the teleconference session, a variety ofother events604 may be issued and acted upon by the conference component. These may include message events, mode events, incoming call events, data transmission events, etc. Members leaving the conference result in the issuance of MemberTerminated (e.g. mtMemberTerminatedEvent) events to the application program such as605 or606. Thus, for every MemberReady event for any member participating in a conference, there is a corresponding MemberTerminated (e.g. mtMemberTerminatedEvent) event prior to the end of the conference. In addition, the auxiliary source and the conference itself is terminated via the Auxiliary Terminated (e.g. mtAuxiliaryTerminatedEvent)event611 and the conference terminatedevent607 as illustrated in600 ofFIG. 6. This notifies the application that the conference is terminated, and teleconference data should no longer be transmitted. Any additional clean up operations are then performed by the application, and the source may be discarded.
A typical application's initialization is shown asprocess700 ofFIG. 7. The application program makes a number of API calls in order to set various parameters associated with the member or potential participant. First, an application may cause the conference component to set its capability atstep702 if it is different than the default. The call to “MTConferenceSetMessageCapabilities” causes the conference component to recall in a store the specific application capabilities within the conference component for the specific conference which are later used during transmission of messages to alert recipients that the sender application has certain functional capabilities prior to the establishment of a connection between the sender and the receiver. Each capability has associated with it a type, version, and “desire” of the capability. Each desire for each capability type may be flagged as:
- 1. optional;
- 2. desired;
- 3. required; or
- 4. a negotiation message.
 These types of capabilities are included in a capabilities list which is transmitted to endpoints, as will be described below. An “optional” capability is a message which is not required to be exchanged before setting up a conference. A “desired” capability is one which is not required that it be exchanged before setting up a conference, however, it is preferred that it is. A “required” capability is one which requires that a message be exchanged before setting up a conference. This may include access control or other messages which are transferred prior to setting up a conference. An access control capability may include the transmission of a account number and password prior to the commencement of a teleconference. A “negotiation message” is a capability which indicates that the application wishes to query the receiving application. In the case of a negotiation message capability, the type field associated with the capability allows the requests of information about the applications prior to the establishment of a conference (e.g. about the type of software and hardware components the application is interested in, such as sound). Any other types of exchanges which require negotiated information between applications may be set.
 
Once all individual capabilities have been set by the issuance of “Set Capabilities” API calls (e.g. MTConferenceSetCapabilities) to the conference component atstep702, a member may set its operating mode atstep704. The mode will be contained within a mode mask value which is sent in the API call to the conference component, and moreover, is included in certain messages transmitted from the conference component in the sender to the conference component in the receiver. The mode mask specifies the characteristics of a conference that the member makes available. Different capabilities, modes, and other initialization operations shown inFIG. 7 may be set for any number of conference types which are made available by the member. At any rate, the default mode includes the following values:
- 1. send media;
- 2. receive media;
- 3. shareable; and
- 4. joiner.
 The “send media” mode flag indicates that the member intends to send media data in its conferences. Most members will want to send media, however, there will be instances where the member will be a receive-only member, thus the send media mode flag will not be set. The receive media mode flag indicates that the member intends to receive media in conferences. In the case of a send-only member (e.g., a server providing a real-time video and/or audio source), will have the receive media mode flag set to “off” (e.g., a numeric value ‘0’). The “shareable” mode flag indicates that the member is willing to share the conference media data with new conference members. Thus, in the instance of a send-only media server, the shareable mode flag would be set indicating that new members can receive the conference data.
 
The “joiner” mode flag indicates that all conference members are allowed to interact. This would allow two-way transmission between each of the conference members. However, the setting of this flag to “off” (e.g., a numeric value ‘0’) results in broadcast type conferences wherein one member sends media data to other conference members, but the individual conference members do not exchange any media data among themselves. Each of these mode flags is transmitted at the beginning of a connection (e.g., contained within the “hello”message1400 inFIG. 14).
By default, the conference component establishes conferences that are fully two-way media data capable, shareable, and joinable. If different characteristics are desired, then the application must call “set mode” (e.g. MTConferenceSetMode) atstep704, along with the appropriate flag(s) set. Conference mode settings are stored and associated with a particular conference ID in the sender's conference component so that when messages are created for that conference ID, the appropriate mode flags are transmitted along with the initialization or other messages sent before any other communications.
In addition to the capabilities and mode settings atsteps702 and704, a time-out value associated with calls placed from the member may be set (e.g. using the API call MTConferenceSetTimeOut). The time-out value is then included at the beginning of certain messages preceding a conference in order to enable a recipient to determine when the caller will stop listening for responses. This allows certain features to be incorporated into participant conference components such as the smart triggering of events based upon context. For example, if the recipient is receiving a call, but a user does not wish to take the call at that time, the recipient's conference knows the time-out occurs and can take certain context-dependent action (e.g., forward the call, send a message, etc.).
The application can then invoke an API call “Listen for Call” (e.g. MTConferenceListen) which implementssteps708 and710. Atstep708, using the underlying transport to which the system is connected, a high level address is registered and published. This then allows other potential participants in the system to call the member. The registration and publication of the address is implementation-dependent, depending upon the media to which the system is connected. Then, atstep710, the conference component waits for incoming calls.
The conference component in the member enters an idle state wherein incoming messages, alerts for the transport component, API and calls will be detected and acted upon. Note that the capabilities, mode, and time-out values are all optional, and the default settings may be used by the member if none of these functions is required by the application. In the call to the MTConferenceListen function, the application must specify the networks on which the member is willing to accept calls. The conference component proceeds to register the member on those networks, doing whatever is appropriate in the various network contexts, and sends an mtListenerStatusEvent to the application to specify whether the registration attempt was successful. After listeners are operational, if another application desires to establish communication with the application, then an mtIncomingCallEvent is forwarded to the application.
FIGS. 8-10 show examples of the various message exchanges between two endpoints. Messages are generated byconference component400 depending upon operating context, and application calls.FIG. 8 shows an example of a calling sequence for setting up a conference between two endpoints. Upon the commencement of a call from a first endpoint such as810 shown inFIG. 8, and a second endpoint such as820 shown inFIG. 8, a “capabilities”message802 is transmitted from theendpoint810 to theendpoint820 if they have been set by the caller. The exchange of “capabilities”messages802 and812 betweenendpoint1810 andendpoint2820 are exchanged after a control channel has been opened on the transmission medium between the participants in an implementation-dependent manner (e.g. by invoking MTConferenceCall). This identifies the capabilities of each of the endpoints, if the capabilities of the application are desired to be transmitted. Once capabilities have been transmitted from each endpoint, each endpoint further transmitsHello messages804 and814 to identify themselves. The details of the capabilities, Hello, and other messages illustrated in the figure will be discussed below. The Hello message is the first step in establishing a conference, and allows conference components in different systems to exchange basic identification and mode information. Subsequent to the exchange of Capabilities messages (if any), and theHello messages804 and814, theendpoint1810 wishing to establish the conference sends acall message806 toendpoint2820. Subsequent thereto, ifendpoint820 desires to engage in the teleconference withendpoint1810, it sends aresponse message816 toendpoint1810. Then, upon the transmission of thecall message806 and the receipt of theresponse message816, a teleconference is then established betweenendpoint1810 andendpoint2820. The details of the process steps performed within each endpoint are discussed with reference toFIGS. 11 and 12.
FIG. 9 illustrates a “Join” protocol process. This is similar to the calling process, however, a “Join”message906 is sent to the second endpoint instead of the “call”message806. The details of a Join are discussed with reference toFIGS. 26a-26c below.
FIG. 10 illustrates the messages exchanged for a terminate message. As illustrated, an endpoint may issue a terminate message to terminate a teleconference. No response is required for any receivers.
FIG. 11 shows the process steps performed by a first participant (e.g.,endpoint1810) wishing to establish communication with a second participant system (e.g.,endpoint2820). First, atstep1102, the caller via implementation-dependent means, accesses the address of the party it wishes to call, either by reference to an internal store, referencing a server or other implementation-dependent manner. Once this has been performed, the application invokes the API call MTConferenceCall in order to call the party atstep1104. Responsive thereto, either a failed event1106 (mtFailedEvent) is received by the caller, or a ringing event1108 (mtPhoneRingingEvent) when the call message has been transmitted to the second participant. In the event of a ringing event, the endpoint can then get the capabilities, mode and the name of the endpoint such as by examining the data contained in theCapabilities message812 and/or theHello message814. Any “required” communication between the caller and receiver may also be performed. Then, the first sender can appropriately configure itself for the second endpoint. Once any necessary message exchanges and/or configurations have been performed in the caller, then the caller will either receive a MemberRefused event1112 (e.g. mtRefusedEvent), for example, if the calling member does not provide the necessary access control subsequent to the call message, or, aMemberReady event1116. Also, a failedevent1106 can be detected at any time, followed by aMemberTerminated event1114. In the case of a MemberRefused event1112, then the conference component will generate aMemberTerminated event1114, and a conference terminated event will then be issued indicating the end of the conference. Once capabilities have been obtained, any required capabilities are checked for at step1113 (e.g. such as any actions that must be performed before the receiver accepts the call). Subsequent thereto, a ConferenceReadyEvent1115 (e.g. MTConfReadyEvent) and a MemberReady event1116 (e.g. mtMemberReadyEvent) can be received by the application, then the endpoint can then engage in the exchange of messages typical during a conference, thus commencing the conference atstep1118.
As shown inFIG. 12, the sequence of steps performed by the receiver is illustrated. For example, atstep1202, the receiver application will detect anincoming call event1202. Subsequent thereto, the receiver can then determine mode, capabilities, caller's name, conference name, and/or return address of the party wishing to engage in the conference. The mode can also be checked for at step1206. Once this has been done, then a time-out check, if any, at1208 can be performed in the receiver. The time-out check may be performed, depending upon an application's implementation, by checking the time-out field shown atfield1504 inFIG. 15, which indicates how long the transmitter will wait prior to timing out and generating a CallFailed event (e.g. mtFailedEvent) in order to terminate the call. In this case, according to implementation, the receiver may do a variety of things, for example, issuing a busy signal to the transmitter, issuing a message to the caller or, taking some action, such as forwarding the call to another party. Thus, the embedding of the time-out field1504, within the initial connection message “Hello,” provides for certain intelligent features in the potential participants of a teleconference. Once any mode checks, if any, have been performed at step1206, then at step1208 a time out check, if any, may be performed. Any user interaction may take place atstep1209. The receiver will then issue a reply atstep1210. The reply may indicate either refusal to answer the call (e.g., due to access control, or the participant already engaging in another conference), or choose to respond to the call in the affirmative indicating that the teleconference may commence. In the case of a refusal, theMemberTerminated1220 event (mtMemberTerminatedEvent) is issued to the application. In the case of the member answering, the MemberReady event1218 (mtMemberReadyEvent) is issued indicating that the medium channel is open and the conference can commence.
Conferencing MessagesConference components exchange a number of messages in order to establish, maintain, and terminate conferences. Conference components also send messages that encapsulate user data, that is, the data that is exchanged by the programs that are using the conference.
After establishing a transport connection but prior to establishing a conference channel with a remote system, a conference member may send either aCapabilities message1300 or anAuxiliary message1700. The member then sends aHello message1400 to identify itself, specifically its mode of operation (send media, receive media, shareable, or joiner) followed by a Call message1500 (to set up a conference) or a Join message1800 (to join to an ongoing conference). The remote member sends aResponse message1600 in answer to the Call or aJoin message1800. Once a conference is established, a member can combine calls or conferences by sending aMerge message1900. Conference members may send or receive a Terminatemessage2300 to end a conference. Connections initially are point-to-point between members of a conference. If the transport medium supports multicast addresses and more than one member is participating in a conference, a broadcast to a multicast address can be used as an optimization. The conference component uses the BroadcastRequest andBroadcastAck messages2400 and2500 to negotiate the transition from point-to-point to multipoint connections using multicast addresses.
All messages supported in the conference messaging protocol are preceded by a 2-byte character identifying the message type. For example for a capabilities message shown inFIG. 13,field1302 contains a ‘K’. All messages are further terminated by a NULL such as that shown infield1308 ofFIG. 13. TheCapabilities message1300 allows a potential member to tell other potential conference members what it can and cannot do in a conference. Each member sends this message before sending the “Hello” message (1400 ofFIG. 14) if capabilities other than the default are supported.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1302 | 2 bytes | ‘K’ | 
|  | delimiter 1304 | 1byte | newline | 
|  | capabilitiesList | 
| 1306 | 0-n bytes | (alphanumeric) | 
|  | terminator 1308 | 1 byte | NULL | 
|  |  | 
capabilitiesList1306- The member's capabilities. This field is optional. If specified, it contains a list of the member's capablilities. An application specifies its capabilities by calling the conference component's MTConferenceSetMessageCapabilities function.
 
TheCapabilities message1300 shown inFIG. 13 informs other potential conference participants about a member's capabilities. These capabilities relate directly to messages the member either supports or requires, one capability for each conferencing message type that the component supports. The messages themselves are defined by the type of application the member is running. For example, video-phone applications and chat lines deliver different services, and use different messages to do so. As a result, the capabilities a member supports will change in light of the application that is participating in the conference. Entries in the capabilitiesList field may request information from the remote system. By setting an entry's “desires” field (2010 ofFIG. 20) mtNegotiationMessageCapabilities (‘N’), a conference member can query for specific information (e.g. about a given component type, such as a codec, hardware/software features supported, etc.). The type field can contain the component type value.
In response to a negotiation Capabilities message, the remote member formats a user data message, for example, containing available component subtypes. Note that this list may contain duplicate entries and entries with a value of NULL. To parse this array, a member must determine the array's size. After sending aCapabilities message1300, the member sends aHello message1400 to establish a conference.
A Hello message (e.g.,1400 ofFIG. 14) is sent at the start of a conference by each endpoint. The Hello message identifies basic capabilities of the sender and the mode in which it will operate. It contains the following:
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1402 | 2 bytes | ‘H’ | 
|  | MinimumVersion 1404 | 1-n bytes | (numeric) | 
|  | delimiter 1406 | 1byte | colon | 
|  | maximumVersion | 
| 1408 | 1-n bytes | (numeric) | 
|  | delimiter 1410 | 1byte | newline | 
|  | conferenceMode | 
| 1412 | 1-n bytes | (numeric) | 
|  | delimiter 1414 | 1byte | newline | 
|  | name | 
| 1416 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1418 | 1byte | newline | 
|  | terminator | 
| 1420 | 1 byte | NULL | 
|  |  | 
minimumVersion1404- The oldest version of the conference protocol supported by the sending component
 maximumVersion1408
- The newest version of the conference protocol supported by the sending component.
 conferenceMode1412
- The desired conference operating mode. This field contains a value that corresponds to the operating mode the sender desires for this conference. Applications specify their desired mode by a SetMode API call (e.g. MTConferenceSetMode discussed above).
 name1416
- The name of the prospective conference member. This name identifies the entity that wants to join the conference, and may represent either an auxiliary data source or a new user. Applications specify a user name by calling the MTConferenceListen API call. The auxiliary name is specified in a MTAttachAuxiliary API call.
 
TheHello message1400 is the first step in establishing a conference. This message allows conference components on different systems to exchange basic identification and capability information. Before sending aHello message1400, a conference component may send either aCapabilities1300 or anAuxiliary message1700. The type of message sent depends upon the role the member anticipates playing in the conference. If the member is looking to join or start a conference, the conference component sends a Capabilities message. If the member is setting up an auxiliary media data source, the component sends anAuxiliary message1700. Following this message, the conference component can enter the call setup phase by sending theCall message1500. If the member wants to provide an auxiliary data source to an existing conference, or join an existing conference, the component sends theJoin message1800.
Callmessage1500 ofFIG. 15 begins the process of establishing a conference connection between two possible participants. This is akin to dialing a number from a phone directory.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1502 | 2 bytes | ‘C’ | 
|  | callTime-out 1504 | 1-n bytes | (numeric) | 
|  | delimiter 1506 | 1byte | tab | 
|  | ConferenceName | 
| 1508 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1510 | 1byte | newline | 
|  | callineConfID | 
| 1512 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1514 | 1byte | newline | 
|  | terminator | 
| 1516 | 1 byte | NULL | 
|  |  | 
callTime-
out1504- The amount of time the calling component is willing to wait for an answer. This field specifies the number of ticks ( 1/60 of a second) the calling component will wait before deciding that the call has not been answered. Called components must respond within this time period. This value may be used by a potential responder for taking some automatic action if the user doesn't answer before the timeout.
 ConferenceName1508
- The conference name. If the caller is establishing a conference, this is the name the caller has assigned to the conference. If the caller is connecting to a conference server, the is the name of the server's conference. Applications set the conference name by calling the MTConferenceCall function.
 callingConfID1512
- The caller's unique conference identifier. This field uniquely identifies the caller's conference endpoint on the network. Conference components create conference identifiers on behalf of calling applications which are unique within the conference component. Call ID's are discussed with reference to2200 ofFIG. 22.
 
TheCall message1500 shown inFIG. 15 begins the process of establishing a conference between two participants. This message can be used in two ways. First, this message can create a conference between two participants. In this scenario, the caller assigns a name to the conference, so that other possible participants may join later. Alternatively, this message can request to join a conference that is managed by a conference server on a network. For example, the server will allow incoming calls, but the function of the server is to merge the new conference due to the call with other ongoing conferences. In other words, the server acts exclusively as a “joiner” and does not supply media data. Once the call is set up, the caller can begin exchanging user data with other conference participants.
Theresponse message1600 ofFIG. 16 is sent in response to Call or Joinmessages1500 or1800.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1602 | 2 bytes | ‘R’ | 
|  | callResponse 1604 | 1-n bytes | (signednumeric) | 
|  | delimiter 1606 | 1byte | newline | 
|  | destinationConfID | 
| 1608 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1610 | 1byte | newline | 
|  | terminator | 
| 1612 | 1 byte | NULL | 
|  |  | 
callResponse1604- The result. This field indicates the result of the preceding Call request. This field is set to ‘0’ if the request was successful. Otherwise, it contains an appropriate result code.
 destinationConfID1608
- The other endpoint's unique identifier. This field identifies the other participant in the conference.
 TheResponse message1600 allows the caller to determine the success of a Call or Join request. TheResponse message1600 indicates how the user at the remote system reacted to the call (for example, whether the user answered the call). ThecallResponse field1604 contains the request's result code. If non-zero, an error occurred and the request was not successful. Otherwise, thedestinationConfID field1608 identifies the endpoint with which the caller may now communicate.
 
Theauxiliary message1700 allows one member to alert the other members of a conference that it is about to provide an auxiliary media data source (a source associated with an ongoing conference). This message may be used in place of theCapabilities message1300 if a participant is being alerted about the presence of an auxiliary media source. The member sends this message before sending the Hello and JoinMessages1400 and1800, and then proceeds to adding an auxiliary data source to the conference.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1702 | 2 bytes | ‘A’ | 
|  | parentConfID 1704 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1706 | 1byte | newline | 
|  | terminator | 
| 1708 | 1 byte | NULL | 
|  |  | 
parentConfID1704- The member's conference identifier. This field identifies the member's existing conference endpoint (the conference identifier the member supplied in the Call message when it first joined the conference). This allows other conference participants to identify the source of the auxiliary data within each participant.
 
TheAuxiliary message1700 informs other conference participants that a member is about to provide an auxiliary conference data source. For auxiliary data sources, this message replace the Capabilities message during early interactions. The member must send this message to each conference participant. The member then sends aHello1400 and aJoin message1800 to each participant. Other participants then accept the new data source by accommodating the Join request.
AJoin message1800 ofFIG. 18 allows a member to join an existing conference, given that conference's identifier. This message is useful for adding auxiliary data sources to an existing conference, and for merging two existing conferences.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1802 | 2 bytes | ‘J’ | 
|  | destinationConfID 1804 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1806 | 1byte | newline | 
|  | callingConfID | 
| 1808 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1810 | 1byte | newline | 
|  | memberlist | 
| 1812 | 0-n bytes | (alphanumeric) | 
|  | terminator 1814 | 1 byte | NULL | 
|  |  | 
destinationConf ID1804- The remote endpoint's unique conference identifier. This field identifies the conference to join on.
 callingConfID1808
- Unique conference identifier. This field uniquely identifies the caller's conference endpoint on the network. Conference components create conference identifiers on behalf of calling applications. If the message is adding an auxiliary media data source, this is the auxiliary's identifier.
 memberList1812
- A list of other conference participants. This list identifies all other known conference participants that are willing to exchange data with new participants (that is, they have the Joiner Mode Mask [e.g. mtJoinerModeMask] set in their conference mode). The conference component can connect with each participant with whom it is not already connected. If the message is adding an auxiliary (via the issuance of an auxiliary message1700), this list contains the endpoint identifier of each participant to which the caller is making the auxiliary available. It is up to each of them to respond.
- This is a list of conference endpoint identifiers; each element in the list is followed by a newline character.
 
TheJoin message1800 allows a member to add an auxiliary data source to an existing conference or to merge two existing conferences. The caller sends this message to members of the conference in response to a merge or join request instead of a call message.
TheMerge message1900 ofFIG. 19 combines two conferences. Recipients of this message connect with the listed members with whom they are not already connected.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 1902 | 2 bytes | ‘M’ | 
|  | conferenceName 1904 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1906 | 1byte | newline | 
|  | myConfID | 
| 1908 | 1-n bytes | (alphanumeric) | 
|  | delimiter 1910 | 1byte | newline | 
|  | memberList | 
| 1912 | 0-n bytes | (alphanumeric) | 
|  | terminator 1914 | 1 byte | NULL | 
|  |  | 
conferenceName1904- The new conference name resulting from the merge. This is the name that was assigned to the conference when the conference was established. Applications specify the conference name by calling the MTConferenceCall API function.
 myConfID1908
- Unique conference identifier. This field uniquely identifies the caller's conference endpoint in the target conference. Conference components create conference identifiers on behalf of calling applications.
 memberList1912
- A list of other conference participants. This list identifies other current conference participants. This list contains the endpoint identifier of each new participant. This is a list of conference endpoint identifiers; each element in the list is followed by a newline character.
 
The Merge message causes the combination of two conferences. This is the mechanism for creating conferences with more than two participants. The caller sends this message to each existing conference member, specifying the conference's name. Each endpoint establishes communications with any new endpoints. By convention, the endpoint with the higher conference identifier value establishes the connection (to avoid duplicate or missed connections). Members of the conference receive aJoin message1900 instead of aCall message1500 when contacted by each new member.
Field SpecificsA capabilities list such as shown inFIG. 20 (e.g., field1306) contains information about the message supported by a conferencing application. The list consists of zero or more lines of information; each line specifies a single capability and consists of the following fields.:
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type 2002 | 4 bytes | (alphanumeric) | 
|  | delimiter 2004 | 1byte | tab | 
|  | version | 
| 2006 | 1-n bytes | (numeric) | 
|  | delimiter 2008 | 1 byte | tab | 
|  | desires 2010 | 1 byte | (alphanumeric) | 
|  | delimiter 2012 | 1 byte | newline | 
|  |  | 
type
2002- Identifies a message by reference to a unique type value.
 version2006
- Message version number. Specifies the most-recent version of the message that the application supports.
 desires2010
- Support level. This field indicates the level of support required by the application. Possible values are:- mtMessageOptionalCapability- Optional. Indicates that the message is optional. The value corresponding to this option is ‘O’.
 
- mtMessageDesiredCapability- Desired. While still optional, this value indicates that the application prefers to receive this message early in the call (e.g. prior to establishing a call, one member may request that the recipient send his business card for long-term storage). The value corresponding to this option is ‘D’.
 
- mtMessageRequiredCapability- Required. The application must receive this message. The value corresponding to this option is ‘R’.
 
- mtNegotiationMessageCapability- Negotiate. The application wants to learn about the recipient's facilities. The value corresponding to this option is ‘N’.
 
 
 
Conference identifiers such as2200 shown inFIG. 22 uniquely identify a conference endpoint. Each endpoint represent a data source or sink for the conference. Note that a single system may have more than one conference endpoint (e.g., an auxiliary) in a given conference, and may have more than one conference at a time. A conference endpoint consists of the following fields:
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | UniqueID | 
| 2202 | 1-n bytes | (numeric) | 
|  | separator 2204 | 1byte | space | 
|  | name | 
| 2206 | 1-n bytes | (alphanumeric) | 
|  |  | 
UniqueID2202- Unique numeric identifier. This field contains a unique numeric endpoint identifier. Each conference component assigns its own identifiers in order to guarantee uniqueness within the context of a given name specified in thename field2206.
 name2206
- A unique name. Identifies the system on the network. The name is unique within the context of a given transport interface. It includes the type of the selected transport and network interface.
 
A member list such as1812 ofFIG. 21 is an array of zero ormore conference identifiers2102,2110, etc. Each entry in the array is delimited by a newline character.
The Terminatemessage2300 ofFIG. 23 ends a conference, closing a member's communications with the participants to which it sends the message.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 2302 | 2 bytes | ‘T’ | 
|  | delimiter 2304 | 1byte | newline | 
|  | terminator | 
| 2306 | 1 byte | NULL | 
|  |  | 
The Terminate
message2300 is the last step in ending a member's participation in a conference. This message ends the member's conference communication with all participants to which it sends the message. If a member is leaving a conference altogether, it must send this message to each conference participant.
TheBroadcastRequest message2400 allows a member to find out if another conference member can accept broadcast (multicast) messages.
|  |  | 
|  | Field | Size | Value | 
|  |  | 
|  | type | 
| 2402 | 2 bytes | ‘B’ | 
|  | subtype 2406 | 1 byte | ‘R’ | 
|  | delimiter 2408 | 1byte | tab | 
|  | multicastAddress | 
| 2410 | 1-n bytes | (alphanumeric) | 
|  | delimiter 2412 | 1byte | newline | 
|  | terminator | 
| 2414 | 1 byte | NULL | 
|  |  | 
subtype2406- The broadcast message subtype. This field must be set to ‘R’.
 multicastAddress2410
- The proposed multicast address. This field contains the multicast address on which the member proposes to send broadcast messages.
 
TheBroadcastRequest message2400 allows a member to determine whether another conference member can accept broadcast messages over a given multicast network address. The recipient indicates its ability to support the broadcast messages by sending a BroadcastAck message2500 (described below). If the recipient cannot support broadcast messaging or cannot access this particular broadcast transmission, the calling member should continue to send point-to-point messages to the recipient.
TheBroadcastRequest message2400 is typically uses as part of the negotiation process that follows merging two conferences or the joining of any new members to a conference. As an optimization, conference participants may choose to set up broadcast capabilities as a more-efficient alternative to maintaining several different point-to-point connections.
TheBroadcastAck message2500 allows a member to respond to aBroadcastRequest message2400.
|  | 
| Field | Size | Value | 
|  | 
| type | 
| 2502 | 2 bytes | ‘B’ | 
| subtype 2504 | 1 byte | ‘A’ | 
| delimiter 2506 | 1 byte | tab | 
| broadcastResponse 2508 | 1-n bytes | (signed numeric) | 
| delimiter 2510 | 1 byte | newline | 
| terminator 2512 | 1 byte | NULL | 
|  | 
subtype
2504- The broadcast message subtype. This field must be set to ‘A’.
 broadcastResponse2508
- The result. This field indicates whether the member can support the multicast address proposed in theBroadcastRequest message2500.
 
TheBroadcastAck message2500 allows a member to indicate whether it can receive messages over a proposed multicast address. Another conference participant proposes a multicast address by sending aBroadcastRequest message2408. If the recipient can support that address, it sets the broadcastResponse field2508 to ‘0’. Otherwise, the broadcastResponse field2580 contains an appropriate non-zero result code. This message is typically used as part of the negotiation process that follows merging two conferences. As an optimization, conference participants may choose to set up broadcast capabilities as a more-efficient alternative to maintaining several different point-to-point connections.
Join operations have the protocol illustrated inFIG. 9.FIGS. 26a-26c illustrate the process steps performed in a transmitter and receiver during join operations.FIG. 26a shows aprocess2600 which includes the process steps performed by a transmitter of a join message or any message containing a member list. This may occur, for example, responsive to an auxiliary or merge message. First, the transmitter creates a join message atstep2602. The destination conference ID and calling conference ID's are added to the message atstep2603. Then, atstep2604, a member is appended to the members list in the join message. This is shown in more detail inFIG. 26b. A transport level connection with the member to receive the join is then performed atstep2605. Subsequent thereto, the message is then sent to any recipients atstep2606.
FIG. 26b illustrates the “append members”function2604 shown inFIG. 26a for appending members to a member list. The function starts atstep2610 which determines whether the member is a “joiner.” If so, then additional members can be appended to the join. If not, then the function ends and the join message with the single member is transmitted as shown onFIG. 26a. At2612, the next member is retrieved according to the conference ID. Atstep2614 it is determined whether there are any more members in the specified conference ID. If not, the process is complete. If there are more members and the shareable mode flag is set, as detected atstep2616, then the member is added to the member list atstep2618. In this manner, during a merge operation, other participants receiving join messages can determine if conference membership has changed, requiring additional join messages to be transmitted by receiving members.
FIG. 26c shows the steps performed by a receiver of a join message. First, atstep2652, the destination conference ID contained in the join message is looked up by the receiver in a locally-stored current conferences history table (e.g.2900 ofFIG. 29). This keeps track of previously used conference ID's for any currently active conferences. If the conference ID has changed, the member can then complete the join by referencing an old conference ID and the current conference ID in the member. This allows for conference merge and auxiliary messages from widely distributed members in a network. If the conference ID can't be found, as detected atstep2654, then the connection is refused atstep2656, by the issuance of an appropriate response message, and the join fails. If the conference ID has been found, then, atstep2660, the connection is added to the list of participants for the conference. Atstep2662, a Response message that the join can be performed is sent to the sender of the join. Then, the membership of the members contained in the member list of the join can then be performed as shown inFIG. 26d.
Themerge membership function2664 is shown in more detail inFIG. 26d. First, it is determined atstep2678 whether the member merging membership is shareable. If so, then atstep2680, it is determined whether there are any more members in the member list. If not, the function is complete. If so, the next member from the membership list is retrieved atstep2682. If the participant is already connected, is the member or is the member's own auxiliary source, as detected atstep2684 then the process returns to step2680. It is determined, atstep2686, whether this party is going to initiate the join with this member in the member list. This prevents conflicting join messages from being received and acted upon in the network. This is accomplished by determining whether the recipient's conference ID is greater than the calling party's conference ID. In this case, this party will take action on the join (e.g. place the call to the other party to accomplish the join). Operations necessary to accomplish the join then take place starting atstep2688. At this step, transport level connections are established. Once established, capabilities, if any, (or an auxiliary message, in the case of an auxiliary source), hello, and join messages are sent atstep2690. The member then waits for a response to the join, and if received in the affirmative, the conference component issues MemberReady to the application. This process continues until all members in the member list have been processed.
Merge OperationsMerge operations are initiated using a “Merge” message (e.g.,1900 ofFIG. 19) which indicates to a participant that two existing conferences should be merged. This is according to the conference ID contained within thefield1908 ofMessage1900. Each Merge message contains within it amemberList1912 which allows the participating members to transmit Join messages to all the members which have been listed in the memberList. This further allows changing membership and conference ID during the course of a merge operation to be tracked so that correct conference ID's and conference membership may be maintained until after the merge. A merge operation is graphically illustrated with reference toFIGS. 26a and 26b. For example, at a first time period as illustrated inFIG. 26a, conference member ID may be engaging in two conferences denoted1 and2 with several other members, A, B, and C. The member then proceeds to issue Merge messages to the members of conferences. That is, member D issues join messages to members A, B and C to Merge the conference2 (denoted B1and C1). At the end of the operation, members A, B, C, and D all have point-to-point communication channels and the new conference ID is the same, (A1, B1, C2, D1, respectively in each of the members). The mechanics of this operation are described briefly with reference toFIGS. 28a-28b. That is, the two separate conferences are now denoted by the same conference ID's in each of the members.
FIGS. 28a-28b show the steps taken during a merge operation by the conference component in the transmitter and receivers (if any) of a merge message.FIG. 28a shows the process steps taken by a transmitter of a merge request. The member first combines the conference ID's of the old and new conference in it's internal store atstep2802. Then, atstep2804, the member creates the merge message and appends each of the members of the conference to the members list in the message via the append members routine2604 (discussed above) in order to ensure full connectivity among all conference members. Then, atstep2806, the merge message is sent to all participants in the merge.
FIG. 28b shows process steps taken in a receiver receiving a merge message. Once a merge message has been detected (step2812), the merge recipient recalls in a local store for example, the conference name and the conference ID atstep2814. Then, as discussed with reference toFIG. 26d, above, a merger of membership is performed (e.g. process2664), and the process is thus complete atstep2816.
In addition to using conference ID's for performing merge and subsequent join operations, contained within each Merge and Join message is a list of members of the conferences being merged or joined. These are included in the memberList fields1812 or1912 ofmessages1800 or1900. For example, due to propagation delays in a networking system, old conference ID's may still exist in peripheral participants, and therefore during merging and or joining operations, conference ID's may become invalid due to members merging conferences, etc. . . . , or reference conference ID's which no longer exist. Thus, contained within each conference component in a member is a current conferences table such as2900 shown inFIG. 29. The current conferences history table allows a member performing a merge or join operation to determine whether in fact conferences to be merged use old conference ID's. If so, then the new conference ID may be used to transmit to the participants receiving the join messages, and/or be matched to the appropriate conference ID. Thus, the member performing the merge operation can reference by way of thecurrent conference ID2901 contained within the merged conferences table, or otherconference ID values2902, containing a variable length list of conference ID's, to which the current conference ID refers. Conference entries are deleted at the end of a conference by the conference component. If a merge occurs at a peripheral location and a conference ID becomes invalid, then the list of conference ID's for an existing conference ID can be referenced by referencing the merged conferences history table.
MulticastFIGS. 30a and 30b illustrate a process which is performed as an optimization if multiple participants within a conference support multicast address broadcast capabilities. First, it is detected atstep3001 whether there are any more transport media to be examined. If so, then the next is retrieved atstep3002, and atstep3003 it is determined whether two or more participants are in the same conference and are on the same transport medium. If so, and the transport medium supports multicast atstep3004, then the multicast capabilities of the transport medium are activated atstep3006. If so, and the multicast capabilities are working atstep3008, then step3008 proceeds to step3012. If it is not working, as detected atstep3008, then the process is aborted atstep3010. At step3012 a multicast address which may be used for the transport medium is retrieved atstep3012. Then, at3014, Broadcast Requests are sent in substantially the format as shown inMessage2400 ofFIG. 24, to all the participants detected atstep3002 supporting multicast. The process then awaits BroadcastAck (in theformat2500 shown inFIG. 25) in order to determine whether the multicast address is available for each atstep3016. If so, then for each BroadcastAck determined to be received atstep3018, and it is determined whether the BroadcastAck was positive (due to the response value contained in field2508 ofFIG. 25). Atstep3022, if the response was positive, then the point-to-point connection is deactivated. Once all the Broadcast Acks have been received (or have timed-out), then it is determined atstep3024 whether there was a Broadcast Ack received for each Broadcast Request message sent. If not, then the process is complete. If so, however,step3026 determines whether there were any positive BroadcastAck for the multicast. If not, then multicast is deactivated, and point-to-point communication continues to be used for communication with the other members (this optimization cannot be performed). The process is thus complete.
Auxiliary OperationsAn auxiliary media source can become part of an ongoing conference. The media source is associated with an ongoing conference by a invoking reference to the main conference stored in each conference component (e.g. by the API call MTConferenceAttachAuxiliarySource), however, it is treated as a separate conference in each conference component. For example, as illustrated with reference toFIG. 31, a first conference referred to byconference ID23B in member B, may be notified of an auxiliary source havingconference ID17A from member A. Note that similar to the protocol illustrated with reference to the Capabilities and Hello messages inFIGS. 8 and 9 above, the Auxiliary message may be sent in place of a Capabilities message between a first endpoint and a second endpoint as illustrated inFIG. 32. The timing of the messages is illustrated with reference toFIG. 32. Similar to the capability and call protocol illustrated with reference toFIG. 8 above, the Auxiliary message3202 is received from thefirst endpoint3210, to notifyendpoint23220 that an auxiliary source is available from the parent conference ID, as specified inparentConfID field1704 ofAuxiliary message1700 ofFIG. 17. Then, aHello message3204 is transmitted fromendpoint13210 toendpoint23220. Responsive thereto,endpoint23220 transmits capabilities message3222 and Hello message3224. Subsequent thereto, a Join message3226 with the conference ID=23B is issued from the endpoint to3220 with the conference ID of the source in order to indicate that theendpoint3220 wishes to receive the auxiliary source AA, shown inFIG. 31. Subsequent thereto, all messages from source A to recipient B as illustrated inFIG. 31 reference thesame conference ID23B for the ongoing conference between A and B. As illustrated inFIG. 31, the “receive media” mode flag is set to not receive (!Receive) and furthermore, the mode of the auxiliary source is not joiner (!Joiner) since it is a receive-only media source. Subsequent thereto, thesecond endpoint3220 sends aresponse message3216 indicating that the auxiliary source has been successfully joined with theconference23B. Subsequent thereto, both media messages for theconference ID23B, and the auxiliary source havingconference ID17A are treated as from the same source (A) by the application. The details and mechanics of a member receiving an auxiliary source message are illustrated with reference toFIGS. 33 and 34.
FIG. 33 shows the process steps performed within a source of an auxiliary source. Atstep3302, the next participant in the conference to which the auxiliary source will be attached is retrieved. It is determined, atstep3304, whether there are any more participants in the conference to which the auxiliary source will be attached. If any, then, atstep3306, auxiliary, hello and join messages are sent to the participant. If accepted via a positive response message (as shown inFIG. 32), the auxiliary then becomes available to the participant for the receiver of the message. The process continues untilstep3304 determines that no additional participants in the conference should join in to monitoring of the auxiliary.
FIG. 34 shows the steps taken by a receiver upon receipt of the initial auxiliary message. Atstep3402, capabilities (if any), and hello are sent. Atstep3404, the auxiliary source media starts to be received, and the conf ID of the parent conference is saved in the conference component of the receiver atstep3406. This allows the application to determine that the auxiliary is associated with the parent media source by storage in the conference component for later retrieval atstep3408. If the auxiliary is received prior to the receipt of a join message, then the association of the auxiliary with the parent conf ID allows the later association of the two from the application's standpoint.
An Example Session (FIGS.35a and35b)An example session using the protocols described above is shown inFIGS. 35a and 35b. The figures illustrate the flow of messages, from the view of a first endpoint. After establishment of the connection between the stations using the transport component, the originating station sendscapabilities message3502, andhello message3504, specifying the calling station's name “James Watt” with the mode mask set to 15 (all bits set—send, receive, joiner, shareable). Then, acall message3506 is transmitted to the receiver along with the timeout value=19392 confname=“some conference,” and the confID=“672 mtlkatlkJamesWatt: VideoPhone: VideoZone”. Simultaneously with the establishment of the connect, corresponding capabilities, hello, and response are sent from the station entitled “Hillary Rodham” as illustrated by packets3508-3512, having the same mode.
Subsequent to the exchange of capabilities, hello, and call and response, amerge message3514 is received by the first endpoint including the conference name “Some Conference”, a conf ID=137 “mtlkatlk Hillary Rodham: Video Phone: Video Zone” and a member list. The endpoint then processes the merge, placing a call sending capabilities, and hello to a member of the old conference, and a join along with the member list as shown bymessages3516,3518, and3520. Once the connection has been established, capabilities andhello messages3524 and3526 are received from the recipient. Responsive to the join the recipient sends aresponse message3528 with the result=0 (request successful). Subsequent thereto, the endpoint then attempts to use a multicast address by transmittingbroadcast request messages3530 and3532. The other members respond withbroadcast Acks3534 and3536, allowing the use of multicast. The conference can then be terminated at any time by any of the members via the transmission of terminate messages to all of the members, such as3538 and3540.
Thus, by use of the foregoing, connections between endpoints, such as for teleconferences between teleconferencing systems, can be enabled. This includes, but is not limited to, the exchange of capabilities and notification of connections between endpoints, the addition of auxiliary data streams to such connections, and the merging of existing connections. It will be appreciated that though the foregoing has been described especially with reference toFIGS. 1-35b, that many modifications made be made, by one skilled in the art, without departing from the scope of the invention as described here. The invention is thus to be viewed as limited only by the appended claims which follow.