US20110249667A1 - Apparatus and method for transmitting media using either network efficient protocol or a loss tolerant transmission protocol - Google Patents

Abstract

A method and apparatus for transmitting voice media over a network where the voice media may be consumed either in a real-time mode or a time-shifted mode. The method comprising transmitting the voice media over the network using a network efficient protocol when either (i) the media is not being consumed in the real-time mode or (ii) the condition on the network is good enough to support the real-time transmission and consumption of the voice media in the real-time mode. Alternatively, the voice media is transmitted using a loss tolerant transmission protocol when the media is being consumed in the real-time mode and the condition on the network is sufficiently poor to prevent the real-time consumption of the voice media in real-time using the network efficient protocol. The apparatus, which may be a communication device or a server, implements the above-described method.

Description

• This application claims the benefit of U.S. Provisional Patent Application No. 61/323,609, filed Apr. 13, 2010, entitled “Communication Services Network and Client Enabled Communication Devices,” which is incorporated herein by reference for all purposes.
BACKGROUND
• 1. Field of the Invention
• This invention relates to communications, and more particularly, to a communication apparatus and method for transmitting media between nodes using either a network efficient transmission protocol or a loss tolerant transmission protocol, depending on (i) the condition on the network and (ii) if either the transmitted media is being consumed in real-time or in a time-shifted mode.
• 2. Description of Related Art
• The Transmission Control Protocol or “TCP” is an example of a network efficient protocol. TCP guarantees the delivery of transmitted data between a sender and a recipient at the expense of speed. For this reason, TCP is currently the most common delivery protocol used on the Internet. A feature called “flow control” is the main reason why TCP is able to guarantee the delivery of media. Flow control determines when data needs to be re-sent and stops the flow of data until previous packets are successfully transferred. For example, when a recipient receives a defective packet or a packet is not received (i.e., a missing packet), a request for retransmission of the defective and/or missing packet is made and flow of subsequent packets is stopped until the retransmission request is satisfied. The guaranteed delivery feature of TCP is beneficial for certain applications, such as the transfer of the content of web pages, files or database information. The possibility that the flow of data may be stopped, however, makes TCP less than ideal for delivery of time critical media, such as streaming voice or video.
• The User Datagram Protocol or “UDP” is an example of a loss tolerant protocol, commonly used on the Internet for streaming audio, video and other time-based media (i.e., media that changes over time). UDP is mainly concerned with the delivery of the most recently available media, as opposed to quality. To achieve the necessary delivery rate for streaming audio or video, there is no form of flow control or error correction with UDP. Without any mechanism for guaranteeing delivery, packets may be received out of order, defective, or lost altogether, possibly resulting in reduced quality of the media delivered to the recipient. As a result when the condition on the network are poor, media may be delivered at a rate sufficient for real-time consumption using UDP when TCP would otherwise be inadequate.
• Conventional communication systems are typically either real-time or time-shifted. Consequently, a protocol like UDP, which is optimized for “real-time” delivery, is typically used for real-time systems, while a protocol like TCP, which is optimized for reliable delivery, is used for time-shifted systems.
SUMMARY OF THE INVENTION
• The invention relates to a method and apparatus for transmitting voice media over a network where the voice media may be consumed either in a real-time mode or a time-shifted mode. The method comprising transmitting the voice media over the network using a network efficient protocol when either (i) the media is not being consumed in the real-time mode or (ii) the condition on the network is good enough to support the real-time transmission and consumption of the voice media in the real-time mode. Alternatively, the voice media is transmitted using a loss tolerant transmission protocol when the media is being consumed in the real-time mode and the condition on the network is sufficiently poor to prevent the real-time consumption of the voice media in real-time using the network efficient protocol. The apparatus, which may be a communication device or a server, implements the above-described method.
BRIEF DESCRIPTION OF THE DRAWINGS
• The invention may best be understood by reference to the following description taken in conjunction with the accompanying drawings, which illustrate specific embodiments of the invention.
• FIG. 1 is an architecture diagram of the communication services network according to the invention.
• FIG. 2 is a block diagram of a client application used in the communication services network of the present invention.
• FIG. 3 is a media flow diagram on a client device running the client application of the present invention.
• FIG. 4 is a diagram of a server architecture used in the communication system of the present invention.
• FIG. 5 is a flow diagram for transmitting media according to the present invention.
• FIG. 6 is a timing diagram illustrating the transmission of media between sending and receiving nodes according to the present invention.
• It should be noted that like reference numbers refer to like elements in the figures.
DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS
• The invention will now be described in detail with reference to various embodiments thereof as illustrated in the accompanying drawings. In the following description, specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art, that the invention may be practiced without using some of the implementation details set forth herein. It should also be understood that well known operations have not been described in detail in order to not unnecessarily obscure the invention.
• The term “media” as used herein is intended to broadly mean virtually any type of media, such as but not limited to, voice, video, text, still pictures, sensor data, GPS data, or just about any other type of media, data or information.
• As used herein, the term “conversation” is also broadly construed. In one embodiment, a conversation is intended to mean a thread of messages, strung together by some common attribute, such as a subject matter or topic, by name, by participants, by a user group, or some other defined criteria. In another embodiment, the messages of a conversation do not necessarily have to be tied together by some common attribute. Rather one or more messages may be arbitrarily assembled into a conversation. Thus a conversation is intended to mean two or more messages, regardless if they are tied together by a common attribute or not.
A. System Architecture
• Referring toFIG. 1, a block diagram of the communication system according to the invention is shown. Thesystem10 includes acommunication services network12, a circuit switched (CS)network14 such as the Public Switched Telephone Network (PSTN), a cellular or mobile phone network, a Push to Talk (PTT) network, or a combination thereof, and agateway16 coupling the twonetworks12,14. Thecommunication services network12 includes one ormore servers18. Thecircuit switch network14 includes, depending on the type of network or combination of networks, one ormore circuit switches22 and possibly one ormore radio towers24.
• Thecommunication services network12 is IP based and layered over one or more communication networks (not illustrated), such as the Public Switched Telephone Network (PSTN), a cellular network based on CDMA or GSM for example, the Internet, an intranet or private communication network, a tactical radio network, a satellite radio network, a first responder network, or any other communication network, or a combination thereof. In various embodiments, thecommunication services network12 is either heterogeneous or homogeneous.
• One or morelegacy communication devices26 are connected to the circuit switchednetwork14 through either wired or wireless connection(s)28, as is well known in the art. In various embodiments, the legacy device(s) may include conventional land-line telephones, mobile or cellular phones, PTT radios, satellite phones or radios, desktop or mobile computers, or any combination thereof.
• One ormore client application30 enabledcommunication devices32 are coupled to thecommunication services network12 through an IP-basednetwork connection34. Depending on the type ofcommunication device32, theconnection34 may be wired (e.g., Ethernet) or wireless (e.g., Wi-Fi, PTT, radio, satellite, CDMA, GSM, etc.). In various embodiments, theclient application30 enabledcommunication devices32 may be any type of telephone, including both land-line, cellular or mobile phones, a PTT radio, satellite based communication device, any type of computer, including but not limited to desktop, laptop, note pad computers, or any other type of wired or wireless communication device.
B. Client Application Architecture
• Theclient application30 is a messaging application that operates in a time-shifted mode, a real-time mode, and provides the ability to seamlessly transition between the two modes. With theclient application30, both inbound and outgoing media is simultaneously and progressively stored as it is either (i) received over anetwork connection34 at thecommunication device32 or (ii) created on thecommunication device32 and transmitted over thenetwork connection34. The storage of the media allows the participants to converse in a time-shifted mode, providing a user experience similar to conventional messaging systems (e.g., email or voice Short Messaging Service (SMS)). The simultaneous and progressive nature of the application
• • enables the participants to converse “live” in a real-time mode, providing a user experience similar to a full-duplex telephone conversation.
• The simultaneous and progressive storage of both transmitted media as it is being created and received media as it is being received further enables a host of rendering options. Such rendering options include, but are not limited to: the real-time rendering of media as the media is received over thenetwork connection34, pause, replay, play faster, play slower, jump backward, jump forward, catch up to the most recently received media, Catch up to Live (CTL), or jump to the most recently received media. As described in more detail below, the storage of media and certain rendering options allow the participants of a conversation to seamlessly transition the conversation from the time-shifted mode to the real-time mode and vice versa. In addition, theclient application30 is capable of supporting multiple types of media, including but not limited to, voice, video, text, still pictures, sensor data, GPS data, or just about any other type of media, data or information.
• Referring toFIG. 2, a logic block diagram of theclient application30 is shown. Theapplication30 includes a Multiple Conversation Management System (MCMS)module40, a Store andStream module42, and aninterface44 provided between the two modules. In one embodiment, theMCMS module40 and the Store andStream module42 communicate throughinterface44 using an encapsulation format (e.g., JSON or XML) over a transport header protocol (e.g., Hypertext Transfer Protocol or HTTP). Theapplication30 functionally is similar to the client application described in U.S. application Ser. Nos. 12/028,400 (U.S. Publication No. 2009/0003558), 12/253,833 (U.S. Publication No. 2009/0168760), 12/253,820 (U.S. Publication No. 20090168759) and 12/253,833 (U.S. Publication No. 2009/0168760), all of which are incorporated by reference herein for all purposes. The individual modules are briefly described below. For more details, the above-listed, co-pending applications should be reviewed.
• TheMCMS module40 includes a number of modules and services for creating, managing and conducting multiple conversations. TheMCMS module40 includes auser interface module40A for enabling the user to interface and control the audio and video rendering and creating functions on thedevice32, rendering/encoding module40B for performing rendering and encoding tasks, acontacts service40C for managing and maintaining information needed for creating and maintaining contact lists (e.g., telephone numbers and/or email addresses), apresence status service40D for both sharing the online status of the user of thedevice32 as well as the online status of the other users on thecommunication services network12. TheMCMS data base40E stores and manages the meta data for messages and conversations conducted using theapplication30 running on adevice32 as well as contact and presence status information. In alternative embodiments, theMCMS database40E may include, but is not limited to, relational databases, file-based databases, object databases, document-oriented databases, or any other type of database and/or database management system that is capable of storing and retrieving data.
• The Store andStream module42 includes a Permanent Infinite Memory Buffer orPIMB46 for storing, in an indexed format, the media of received and sent messages. The store andstream module42 also includes an encode-receivemodule42A, net receivemodule42B, transmitmodule42C and a rendermodule42D. The encode-receivemodule42A performs the function of receiving, encoding, indexing and storing in a time-indexed format in thePIMB46 media created using theclient application30 ondevice32. The net receivemodule42B performs the function of indexing and storing in the time-indexed format in thePIMB46 the media contained in messages received fromother devices32 or26 through agateway client16. The transmitmodule42C is responsible for both storing in the PIMB and transmitting to recipients the media of messages created using thedevice32. The rendermodule42D enables theclient application30 to render ondevice32 the media of messages, either in the near real-time mode as media is received over thenetwork12 or in the time-shifted mode by retrieving and rendering the media stored in thePIMB46.
• TheMCMS module40 and the Store andStream module42 also each communicate withvarious hardware components48 provided on thedevice32, including, but not limited to, encoder/decoder hardware48A,network interface48B for connecting thedevice32 tonetwork connection34, andmedia drivers48C. The encoder/decoder hardware48A is provided for encoding the media, such as voice, text, video or sensor data, generated by a microphone, camera, keyboard, touch-sensitive display, GPS, sensor, etc. provided on or associated with thedevice32 and decoding similar media before it is rendered on thedevice32. Themedia drivers48C are provided for driving the media generating components, such as speaker and/or a display (not illustrated) after the media has been decoded. Thenetwork interface48B is provided for the connectingdevice32 to anetwork connection34, either through a wireless or wired connection. Although not illustrated, theclient application30 runs or is executed by an underlying processor embedded indevice32, such as a microprocessor or microcontroller.
• The transmitted and received media stored in thePIMB46 is persistently stored. The term persistent storage as used herein is intended to have broad meaning. In various embodiments, persistent storage is intended to mean the storage of media and meta data from just beyond transient storage needed to either transmit or render media in real-time to storage for an indefinite period of time. The term persistent storage therefore may have different meanings, depending specific implementations or embodiments.
• In addition, “real-time” is intended to mean the consumption or rendering of time-based media (i.e., media that changes over time) as the media is being transmitted, regardless if the media is “live” or not. The real-time consumption of “live” media is intended to mean the rendering of time-based media as the media is being created and transmitted. The real-time consumption of non-live media is intended to mean the consumption of previously recorded time-based media that is being transmitted out of storage.
• Referring toFIG. 3, a media flow diagram on acommunication device32 running theclient application30 is shown. The diagram illustrates the flow of media for both the transmission and receipt of media, in either the real-time mode or the time-shifted mode.
• (i) The Receipt of Media: Media received from thecommunication services network12 is simultaneously and progressively stored in thePIMB46 by the net receivemodule42B as the media is over thenetwork connection34, as designated byarrow50, regardless if the media is to be rendered in real-time or in the time-shifted mode. When in the real-time mode, the media is also simultaneously and progressively provided by the net receivemodule42B, as designated byarrow52, to the rendermodule42D. In response, the rendermodule42D simultaneously and progressively renders the media as it is received overconnection34 on a media-rendering device (e.g., a speaker and/or display). In the time-shifted mode, the media is retrieved by the rendermodule42D, as designated byarrow54, from thePIMB46 at an arbitrary time after it was persistently stored. The retrieved media is then rendered on the media rendering devices, such as a speaker and/or display. In this manner, the recipient of the media may review persistently stored media at any time after storage in the time-shifted mode.
• (ii) Transmitting Media: Media created ondevice32 by a media creating device (e.g. a microphone, keyboard, video and/or still camera, a sensor such as a thermometer or GPS, or any combination thereof) is progressively stored in thePIMB46 in a time-indexed format as it is created, as designated byarrow58. In most situations, the media is also provided, as designated byarrow56, to the transmitmodule42C, which simultaneously and progressively transmits the media as it is created. In other situations, media may be transmitted by transmitmodule42C out of thePIMB46 at some arbitrary time after it was created, as designated byarrow60. Transmissions out of thePIMB46 typically occur when media is created when thedevice32 is disconnected from thenetwork12. When thedevice32 reconnects, the media is read from thePIMB46 and transmitted by the transmitmodule42C.
• As a clarification, the media creating devices (e.g, microphone, camera, keyboard, etc.) and media rendering devices (e.g., speaker and display) as illustrated inFIG. 4 are intended to be symbolic of such devices. It should be understood such devices are typically embedded incommunication devices32, such as mobile or cellular phones, radios, mobile computers, etc. With other types ofcommunication devices32, such as desktop computers, the media rendering or generating devices may be either embedded in or plug-in accessories.
C. Communication Services Network
• Referring toFIG. 4, a block diagram of thecommunication services network12 is shown. As noted above, thenetwork12 includes one ormore servers18. In a non-exclusive embodiment, eachserver18 includes one ormore routers20, one ormore header stores62, and one or more body stores64.
• Therouters20 communicate withother routers20, toheader stores62 for read and/or write operations, tobody stores64 for read and/or write operations.Routers20 are further responsible for updating routing tables and maintaining the presence status information of users on thenetwork12.Routers20 also perform a number of security functions, including authentication, encryption, and authorization.
• In a non-exclusive embodiment, the one ormore servers18 on thenetwork12 are highly configurable and scalable. For example, if a large number of users subscribe to the services provided by thenetwork12, then a large number ofrouters20 may be needed. If aserver18 routes a high volume of traffic, but the messages tend to be relatively short in duration (e.g., contain minimal media), then the number ofheader stores62 may be increased relative to the number of body stores64. Alternatively, if the traffic handled by a server tends to have large amounts of media (i.e., the messages are long in duration), thenmore body stores64 may be needed. Further, the number ofservers18 included on the network may be increased or reduced as needed.
• On thenetwork12, each of the server(s)18 subscribe to all of the header and body data for a given user and/or group of users. As a result, if aserver18 that holds the header and/or body information for a user becomes unavailable, arouter20 may be able to locate anotherserver18 to obtain the data. In other embodiments, one or more users, servers or any other entity may subscribe based on the domain of user(s), defined sets of users, media type, codec type used to encode the media, conversation name, conversation subject or topic, time range, or any other type of defined criteria.
D. Vox Messages and Media Payloads
• Client application30 enableddevices32 communicate with one another using individual message units, referred to herein as “Vox messages”. By sending Vox messages back and forth over thecommunication services network12, the users of thedevices32 may communicate with one another, either in the real-time mode or in a time-shifted messaging mode, and with the ability to seamlessly transition between the two modes.
• There are two types of Vox messages, including (i) messages that do not contain media and (ii) messages that do contain media. Vox messages that do not contain media are generally used for message meta data, such as media headers and descriptors, contacts information (telephone numbers or email addresses), presence status information, etc. Message meta data includes such attributes as a message identifier or ID, the identification of the message originator, a recipient list, and a message subject. The identifier information may be used for a variety of reasons, including, but not limited to, building contact lists, associating media with messages, and/or associating messages with conversations. The Vox messages that contain media are used for the transport of media. In one embodiment, messages containing text media may include both meta data and the text media, whereas messages containing time-based media, such as voice or video, do not contain meta data.
• In one embodiment, Vox messages are layered on top of the application layer of whatever transport protocol or protocols are used on the underlying network infrastructure below thenetwork12. As a result, a new transport protocol for Vox messages is not needed. Rather, Vox messages are transmitted and routed across thenetwork12 using current transport protocols running over the existing telecommunications infrastructure.
• The presence status information contained in Vox messages may be used to identify the users that are currently authenticated by thesystem10 and/or if a given user is reviewing a message in real-time or not. The presence data is therefore useful in determining, in certain embodiments, how messages are delivered across thenetwork12. In situations where the presence status indicates an authenticated user is reviewing a message in real-time for example, then a transport protocol optimized for timely (i.e., real-time) delivery, such as UDP, may be used, whereas a transport protocol optimized for efficient delivery of messages, such as TCP, may be used when the presence status indicates the authenticated user is not reviewing the message in real-time.
E. Transmission Protocols and Complete Copies of Media
• Referring toFIG. 5, a flow diagram80 for transmitting media between nodes on thecommunication services network12 according to the present invention is shown. The sending and receiving node pair may be between acommunication device32 and aserver18, betweenserver18 hops on thenetwork12, or between any two nodes on the network. As described below, either a network efficient protocol or a loss tolerant protocol may be used, depending on (i) the condition on the network and (ii) if the media is being consumed in real-time by the recipient.
• In theinitial decision82, a transmission loop is defined at the sending node and the sending node determines if there is any media available for transmission. If not, step82 is continually repeated until media becomes available for transmission. When media is available, it is next determined (decision84) if the media is or will be consumed in real-time based on the presence information of the recipient(s). If the presence information of all the recipient(s) indicates none are reviewing in real-time, then the media is transmitted using a network efficient protocol (step86). If one or more of the recipients is reviewing the media in real-time, then the transmitting node determines if the condition on the network (decision88) is sufficient for transmitting the media at a rate sufficient to support real-time consumption using the network efficient protocol.
• If the condition on the network is sufficient for supporting real-time communication using the network efficient protocol, then the transmitting node continues transmitting using the network efficient protocol (step86). If the condition is not sufficient, however, then the transmitting node transmits the media using a loss tolerant protocol (step90).
• With media that is transmitted using the loss tolerant protocol, it is determined indecision92 if the condition on the network is sufficiently good enough (i.e., the rate of media loss is minimal) to support real-time communication. If the condition on the network is not sufficient, meaning real-time communication is not possible or practical using even the loss tolerant protocol, then the transmitting node stops using the loss tolerant protocol. Instead, the media is transmitted using the network efficient protocol (step86) as the condition of the network permits. On the other hand if the condition on the network is sufficiently good enough to support real-time communication, then the media is transmitted using the loss tolerant protocol.
• Indecision94, it is determined if the message has ended or if the recipient is no longer reviewing in the real-time mode. If neither condition is met, then another transmission loop is defined (decision82) and the above process is repeated. When either condition is met, the media originally transmitted using the loss tolerant protocol is retransmitted when network conditions permit using the network efficient protocol, which guarantees the eventually delivery of a complete copy of the media as originally encoded. In most situations, the retransmission occurs when bandwidth on the network is available beyond what is needed to support real-time communication. In one embodiment, a complete copy of the media previous sent using the loss tolerant protocol is retransmitted. In an alternative embodiment, just the missing media is transmitted.
• The aforementioned process is continually repeated while media is available for transmission. With each cycle, a transmission loop is defined and the above-described process is repeated.
• The transmission protocol as described above with respect toFIG. 5 resides in layer above or on top of the underlying network efficient and loss tolerant protocols. In one embodiment, the network efficient protocol is used as the default protocol, meaning when a transmission begins, the network efficient protocol is initially used and the loss tolerant protocol is used only if the media is being consumed in real-time and the condition on the network is not good enough to support real-time consumption. In an alternative embodiment, the loss tolerant protocol may be the protocol that is initially used when a transmission begins and the network efficient protocol is used only when either (i) the media is not being consumed in real-time and/or (ii) the condition on the network is good enough to support real-time communication using the network efficient protocol. The second embodiment is typically used in situations when it is known that the transmitted media is or will likely be consumed in real-time and/or the condition on the network is typically inadequate to support real-time consumption.
• In a specific embodiment using TCP and UDP, the transmission protocol as described above with respect toFIG. 5 monitors the rate at which the TCP protocol transmits media. So long as the transmit buffer used by the TCP protocol does not back-up beyond a predetermined threshold or overflow, TCP is used for transmissions, regardless if the media is being consumed in real-time or not. If the transmission buffer backs-up beyond the predetermined threshold level or overflows, indicating that the condition on the network is inadequate to support the transmission of media at a rate sufficient to support real-time consumption, then a switch occurs and UDP is used for media that is being consumed in real-time. The transmitting node will continue using UDP until (i) the transmitted media is no longer being consumed in real-time, (ii) the condition on the network has improved to the point where TCP may be used for the transmission and consumption of the media in real-time; and/or (iii) conditions on the network are so poor, it is not practical for the media to be consumed in real-time even using UDP.
• The transmission protocol as described above with respect toFIG. 5 resides at bothcommunication devices32 and atservers18 on thenetwork12. In a non-exclusive embodiment, the transmission protocol is embedded in theclient application30. In an alternative embodiment, the transmission protocol may be separate from but cooperate with theapplication30. Regardless of how implemented, the transmission protocol as described herein allows bothclient30communication devices32 and server(s)18 to communicate with one another using either a loss tolerant or network efficient protocol, depending on if the media being transmitted is being consumed in real-time and the condition of the network.
• Referring toFIG. 6, a timing diagram illustrating the transmission of media between sending and receiving nodes is illustrated. Initially at time T1,client32A begins transmitting the media of a message (step82) torecipient client32B using the network efficient protocol (e.g., TCP). At time T2, the presence information of therecipient32B indicates if the media of the message is being reviewed in real-time or not. Based on the presence information (decision84), the sending node evaluates if the media is being consumed in real-time or not. In this timing diagram example, it is assumed that the media is in fact being consumed in real-time and the network is not good enough (decision88) to support real-time communication using the network efficient protocol. As a result at time T3, the media of the message is transmitted using the loss tolerant protocol (step90). Eventually the message ends and/or the recipient transitions from the real-time to the time-shifted mode. When either event occurs, as designated by the time T4, the transmittingclient32A retransmits, according to different embodiments, either just the missing media or the media previously sent using the loss tolerant protocol. In either case, the network efficient protocol is used, guaranteeing the delivery of a complete copy of the media of the message to therecipient32B.
• The timing diagram is intended to illustrate the asynchronous nature of the transmission flow diagram illustrated inFIG. 5. Many of the steps and/or decisions outlined in the flow diagram ofFIG. 5 occur in parallel to the extent possible. Many of the events as described overlap with one another to some extent and do not necessarily occur in “lock-step”, meaning it is not necessary for one step to start only after the previous step is complete, as is typical with most synchronous “clock-driven” communication systems. Rather, each step is started as soon as the conditions necessary to perform that step are completed. This is particularly true with the transmission of media for example. Conventional communication systems typically transmit media using only a single transmission protocol. With the protocol as described herein, however, the transmission of media may be switched “on the fly” during the course of a message between either the network efficient or loss tolerant protocols, depending on the presence status of the recipient(s) and/or condition on the network.
• The transmission of media betweencommunication devices32, as described above, occurs through one ormore servers18 on thenetwork12. In an alternative pier-to-pier embodiment however, it would be possible for twocommunication devices32 to communicate directly with one another. With this embodiment, the flow and timing diagrams ofFIGS. 5 and 6 would still be applicable, except the communication flow would occur directly over thenetwork12 between a sendingcommunication device32 and one or morerecipient communication devices32.
• It also should be noted that for the sake of simplicity, the transmission of messages as described above has been “one-way”. It should be understood that the transmissions of media, using either the network efficient and/or the loss tolerant protocol, is often bi-directional or “two-way”. With two-way communication in the real-time mode, the user experience is similar to a conventional full-duplex conversation. In addition, bi-directional communication can also take place between multiple parties (i.e, more than two), similar to a multi-party conference call.
• By using either the network efficient or loss tolerant protocol, depending on if media is being consumed in real-time and/or on the condition of the network, transmissions are optimized for real-time when needed or for efficient delivery when real-time delivery is not critical or network conditions are sufficiently good to support real-time communication using the network efficient protocol. On the other hand when the media is being reviewed in real-time and network conditions are poor, the loss tolerant protocol is typically used to extend or enhance the ability to conduct real-time communication. Since any missing or media transmitted using the loss tolerant protocol is eventually retransmitted using a network efficient protocol that guarantees the delivery, the recipient eventually receives a complete copy (i.e. a full bit rate representation as the media was originally encoded). The full bit rate representation of the media typically replaces any missing, defective or out of order representations of media previously received and stored in thePIMB46 of the receivingdevice32. In this manner, the recipient may review the complete, full quality, version of the media in the time-shifted mode at a later arbitrary time.
• In one embodiment, TCP is used as the network efficient protocol, while UDP is used as the loss tolerant protocol. In other embodiments, TCP is used as the network efficient protocol, while the Cooperative Transmission Protocol (CTP) described in U.S. application Ser. No. 12/192,890, incorporated by reference herein, is used as the loss tolerant protocol. In other embodiments, any network efficient or loss tolerant protocol may be used.
• Although many of the components and processes are described above in the singular for convenience, it will be appreciated by one of skill in the art that multiple components and repeated processes can also be used to practice the techniques of the system and method described herein. Further, while the invention has been particularly shown and described with reference to specific embodiments thereof, it will be understood by those skilled in the art that changes in the form and details of the disclosed embodiments may be made without departing from the spirit or scope of the invention. For example, embodiments of the invention may be employed with a variety of components and should not be restricted to the ones mentioned above. It is therefore intended that the invention be interpreted to include all variations and equivalents that fall within the true spirit and scope of the invention.

Claims (50)

1-17. (canceled)

18. A method for transmitting media from a node to one or more recipients over a network using either a network efficient protocol or a loss tolerant protocol, the method comprising:

(a) ascertaining if media available for transmission is or will be consumed in real-time by any of the one or more recipients; and either:

(b) transmitting the media from the node over the network to the one or more recipients using the loss tolerant protocol when:

(i) at least one of the one or more recipients is or will be consuming the media in real-time; and

(ii) the available bandwidth on the network is not good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol; or

(c) transmitting the media from the node over the network to the one or more recipients using the network efficient protocol when:

(iii) none of the one or more recipients is or will be consuming the media in real-time; or

(iv) one or more of the recipients is or will be reviewing the media in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol.

19. The method ofclaim 18, wherein the network efficient protocol is TCP.

20. The method ofclaim 18, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

21. The method ofclaim 18, wherein the loss tolerant protocol is UDP.

22. The method ofclaim 18, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

23. The method ofclaim 18, further comprising:

identifying the media transmitted using the loss tolerant protocol; and

re-transmitting at least a portion of the identified media using the network efficient protocol.

24. The method ofclaim 23, wherein re-transmitting the at least a portion of the identified media comprises one of the following:

(i) re-transmitting the identified media in full; or

(ii) re-transmitting just the defective or media lost when transmitting using the loss tolerant protocol.

25. The method ofclaim 23, wherein the identified media is re-transmitted when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

26. The method ofclaim 18, further comprising repeatedly defining transmission loops and performing (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

27. The method ofclaim 18, further comprising determining from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

28. The method ofclaim 18, wherein the media is time-based media that changes over time.

29. The method ofclaim 18, wherein the consumption of the media in real-time further comprises one of the following:

(i) the rendering of the media “live” as the media is created and transmitted; or

(ii) the rendering of previously created and stored media streamed from storage.

30. The method ofclaim 18, further comprising:

defining the network efficient protocol as a default protocol when initially transmitting the media; and

switching the transmission of the media from the network efficient protocol to the loss tolerant protocol when:

the media is or will be consumed in real-time; and

the available bandwidth on the network is not good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol.

31. The method ofclaim 18, further comprising:

defining the loss tolerant protocol as a default protocol when initially transmitting the media; and

switching the transmission of the media from the loss tolerant protocol to the network efficient protocol when either:

(i) the media is not being consumed in real-time; or

(ii) the media is being consumed in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol

32. The method ofclaim 18, further comprising ascertaining the bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission at the node backs-up beyond a predetermined threshold or overflows during the transmission of the media.

33. The method ofclaim 18, wherein the media comprises one of the following: voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

34. A communication application embedded in a non-transient computer readable medium and intended to run on a communication device, the application comprising:

a transmission module, the transmission module configured to:

(a) ascertain if media available for transmission is or will be consumed in real-time by any of one or more recipients; and either:

(b) transmit the media over a network to the one or more recipients using the loss tolerant protocol when:

(i) at least one of the one or more recipients is or will be consuming the media in real-time; and

(ii) the available bandwidth on the network is not good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol; or

(c) transmit the media over the network to the one or more recipients using the network efficient protocol when:

(iii) none of the one or more recipients is or will be consuming the media in real-time; or

(iv) one or more of the recipients is or will be reviewing the media in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol.

35. The application ofclaim 34, wherein the network efficient protocol is TCP.

36. The application ofclaim 34, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

37. The application ofclaim 34, wherein the loss tolerant protocol is UDP.

38. The application ofclaim 34, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

39. The application ofclaim 34, wherein the transmission module is further configured to:

identify the media transmitted using the loss tolerant protocol; and

re-transmit at least a portion of the identified media using the network efficient protocol.

40. The application ofclaim 39, wherein the transmission module is further configured to re-transmit by performing one of the following:

(i) re-transmitting the identified media in full; or

(ii) re-transmitting just media lost when transmitting using the loss tolerant protocol.

41. The application ofclaim 39, wherein the transmission module is further configured to re-transmit when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

42. The application ofclaim 34, wherein the transmission module is further configured to repeatedly define transmission loops and perform (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

43. The application ofclaim 34, wherein the transmission module is further configured to determine from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

44. The application ofclaim 34, wherein the media is time-based media that changes over time.

45. The application ofclaim 34, wherein the consumption of the media in real-time comprises one of the following:

(i) the rendering of the media “live” as the media is created and transmitted; or

(ii) the rendering of previously created and stored media streamed from storage.

46. The application ofclaim 34, wherein the transmission module is further configured to:

define the network efficient protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the network efficient protocol to the loss tolerant protocol when:

the media is or will be consumed in real-time; and

the available bandwidth on the network is not good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol.

47. The application ofclaim 34, wherein the transmission module is further configured to:

define the loss tolerant protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the loss tolerant protocol to the network efficient protocol when either:

(i) the media is not being consumed in real-time; or

(ii) the media is being consumed in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol

48. The application ofclaim 34, wherein the transmission module is further configured to ascertain a bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission backs-up beyond a predetermined threshold or overflows during the transmission of the media.

49. The application ofclaim 34, wherein the media comprises one of the following:

voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

50. A communication device, comprising:

a transmission element, the transmission element configured to:

(a) ascertain if media available for transmission is or will be consumed in real-time by any of one or more recipients; and either:

(b) transmit the media over a network to the one or more recipients using the loss tolerant protocol when:

(i) at least one of the one or more recipients is or will be consuming the media in real-time; and

(ii) the available bandwidth on the network is not good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol; or

(c) transmit the media over the network to the one or more recipients using the network efficient protocol when:

(iii) none of the one or more recipients is or will be consuming the media in real-time; or

(iv) one or more of the recipients is or will be reviewing the media in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol.

51. The device ofclaim 50, wherein the network efficient protocol is TCP.

52. The device ofclaim 50, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

53. The device ofclaim 50, wherein the loss tolerant protocol is UDP.

54. The device ofclaim 50, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

55. The device ofclaim 50, wherein the transmission element is further configured to:

identify the media transmitted using the loss tolerant protocol; and

re-transmit at least a portion of the identified media using the network efficient protocol.

56. The device ofclaim 55, wherein the transmission element is further configured to re-transmit by performing one of the following:

(i) re-transmitting the identified media in full; or

(ii) re-transmitting just media lost when transmitting using the loss tolerant protocol.

57. The device ofclaim 55, wherein the transmission element is further configured to re-transmit when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

58. The device ofclaim 50, wherein the transmission element is further configured to repeatedly define transmission loops and perform (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

59. The device ofclaim 50, wherein the transmission element is further configured to determine from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

60. The device ofclaim 50, wherein the media is time-based media that changes over time.

61. The device ofclaim 50, wherein the consumption of the media in real-time comprises one of the following:

(i) the rendering of the media “live” as the media is created and transmitted; or

(ii) the rendering of previously created and stored media streamed from storage.

62. The device ofclaim 50, wherein the transmission element is further configured to:

define the network efficient protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the network efficient protocol to the loss tolerant protocol when:

the media is or will be consumed in real-time; and

the available bandwidth on the network is not good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol.

63. The device ofclaim 50, wherein the transmission element is further configured to:

define the loss tolerant protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the loss tolerant protocol to the network efficient protocol when either:

(i) the media is not being consumed in real-time; or

(ii) the media is being consumed in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol

64. The device ofclaim 50, wherein the transmission element is further configured to ascertain a bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission backs-up beyond a predetermined threshold or overflows during the transmission of the media.

65. The device ofclaim 50, wherein the media comprises one of the following: voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

66. The apparatus ofclaim 50, wherein the communication device comprises one of the following: a cellular phone, a mobile phone, a land-line phone, a PTT radio, a satellite radio, a desktop computer, a mobile computer, a wired communication device, a wireless communication device or a server.

Images