US20040082316A1 - Centralized server methodology for combined audio and video content - Google Patents

Abstract

A teleconferencing system with capability to store incoming multiple medium messages for later retrieval and playback is disclosed. The system includes a communications multiplexer which, in normal mode, receives the incoming message and routes the message to various output functions, including video, audio, and computer display. In store mode, the communications multiplexer receives the incoming message and communicates it to disk storage, for example by way of direct memory access. During playback, the communications multiplexer receives data from the disk storage, and controls its communication to the various output functions, in the same manner as during receipt of a normal incoming message during an interactive teleconference. As a result, multiple medium messages may be stored for later retrieval, with the playback appearing in the same manner as a conventional teleconference message.

Description

• This invention is in the field of telecommunications, and is more specifically directed to communication systems using multiple media.[0001]
BACKGROUND OF THE INVENTION
• In the business world, face-to-face meetings generally include the exchange of information by way of various media. Obvious modes of interpersonal information exchange include voice and visual communication. In addition, documents are often exchanged or examined by the meeting participants, slides or viewgraphs may be shown and discussed in presentations, and drawings are often made on paper or a board, accompanied by an oral explanation. Modern meetings and conferences also can involve discussion of data processed by a computer, such as the results of spreadsheet calculations. The ability to communicate one's point using the most effective medium available, together with the opportunity to observe participants' reactions, cause many people to favor face-to-face meetings over communication by mail, FAX, or telephone.[0002]
• Systems have recently become available which enable the conducting of meetings and conferences over long-distances, communicating not only audio information but also video, documents, graphics, and other, types of information. Complex systems for accomplishing such meetings include satellite or other closed-circuit television techniques. Such television-based systems are relatively costly to set up and operate, primarily due to the costs of transmission of the information.[0003]
• The speed, accuracy and reliability of communication over telephone lines, particularly digital telephone networks, has now improved to the extent that video and graphics information can be transmitted thereover, at sufficiently high rates that real-time teleconferencing can include audio, video, and computer graphics information. The ability to communicate multiple medium presentations has made video teleconferencing much more attractive to the participants, reducing the need of personnel to travel or relocate for effective communications and management.[0004]
• An example of a modern teleconferencing system is the CS420R system manufactured and sold by VideoTelecom Corporation of Austin, Tex. This system provides real-time multiple-way communication of multiple medium information over conventional digital telephone networks. The media include audio and video, with the video information generated by the digitization of the output of a video camera. Other information communicated by this system includes computer-generated data, for example spreadsheet data, computer graphics, and interactive graphics data input by way of a graphics pad, mouse, or other input device. Documents can also be communicated in real-time according to this system, by way of facsimile, so that the conference participants may refer to the same document during the meeting. Other media include digitized still images, digital information from CD-ROM, and other known information media in the industry. Additional technical detail regarding the CS420R system is provided in Duran, et al., “A PC-compatible, multiprocessor workstation for video, data, and voice communication”,[0005]Visual Communications&Image Processing IV, William A. Perlman, ed., (SPIE, 1989), pp. 232-236, incorporated herein by reference.
• As in the case with telephonic communication, however, an interactive conference requires that the parties each be connected to the system at the same time. Scheduling conflicts, time zone differentials, and other events which prevent parties from meeting face-to-face, also preclude meeting by phone or teleconference. These factors have presented the need for storage and retrieval of messages. Currently available systems for storage of audio messages include simple answering machines and complex voice mail systems, are now widely available. Another example of a well-known storage system for long-distance messages is a computer bulletin board, upon which various types of computer data may be stored. Of course, these systems are each limited to the storage and delivery of a single type of message. Particularly in today's complex worldwide business climate, it is desirable to transmit and store full multiple-media presentations for later retrieval.[0006]
• It is therefore an object of this invention to provide a system for transmission of a multiple medium message, and for storage of the transmitted message for later retrieval.[0007]
• It is a further object of this invention to provide such a system where the media include audio, video, graphics, and any other information which may be acquired or generated by a computer and stored in computer-accessible memory.[0008]
• It is a further object of this invention to provide such a system which utilizes existing teleconferencing hardware.[0009]
• Other objects and advantages of the invention will become apparent to those of ordinary skill in the art having reference to the following specification together with its drawings.[0010]
SUMMARY OF THE INVENTION
• The invention may be implemented into a teleconferencing system which receives and stores digital information representative of information from multiple media. The system includes a communications multiplexer which, during normal transmission, routes incoming information to the proper output processors and devices in the system. The system is programmable to receive and store an incoming message by controlling the communications multiplexer to route the incoming information to disk storage. In playback, the communications multiplexer receives the stored information, and routes the appropriate portions thereof to the proper output devices in a similar manner as in normal transmission.[0011]
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is an electrical diagram, in block form, of a conventional two-way teleconferencing system and its various input and output devices.[0012]
• FIG. 2 is an electrical diagram, in block form, of a computer in the system of FIG. 1, according to the preferred embodiment of the invention.[0013]
• FIG. 3 is an electrical diagram, in block form, of the communications multiplexer in the computer of FIG. 2.[0014]
• FIG. 4 is a flow chart illustrating the operation of the system of FIGS. 1 and 2 in storing a transmission for later retrieval.[0015]
• FIG. 5 is a flow chart illustrating the steps of storing data corresponding to the transmission.[0016]
• FIGS. 6[0017]aand6bare electrical diagrams of the communications multiplexer of FIG. 3, illustrating the flow of data in the steps of FIG. 5.
• FIG. 7 is a flow chart illustrating the operation of the system of FIGS. 1 and 2 in retrieving a stored transmission.[0018]
• FIG. 8 is a flow chart illustrating the steps of retrieving data corresponding to the stored transmission.[0019]
• FIGS. 9[0020]aand9bare electrical diagrams of the communications multiplexer of FIG. 3, illustrating the flow of data in the steps of FIG. 8.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
• Referring now to FIG. 1, a teleconferencing system according to the preferred embodiment of the invention will be described in the context of a two-party video teleconference. It should be understood, however, that multiple party teleconferences are also commonplace and will benefit from the present invention. A[0021]video teleconferencing system1, such as the CS420R system noted above, is located at each of locations A and B of FIG. 1. Connection between the systems at locations A and B is made by way of a conventional digital communications network DCN, as is prevalent in the world today; accordingly, locations A and B may be in different cities, or even different countries.
• Each[0022]system1 includes amain computer2, which connects to network DCN by way ofconventional interface4.Computer2, in the CS420R system noted hereinabove, is configured in the chassis of a conventional personal computer such as an IBM PC AT, but includes significant additional video and other data processing capability.Computer2 further includes communications hardware, as will be described in further detail hereinbelow, for interfacing to the various input and output devices at each location. It should be noted that modern output devices often include audio or video decoding, and also processing capability for performing other types of data processing, some of which may be quite complex. In the example of FIG. 1, eachsystem1 includes CRT monitors6 for displaying both video from the other system and another image, such as computer graphics; in a “preview” mode, monitors6 may also display information for transmission from itsown system1. Alternatively, different visual output, such as still images, motion video, computer graphics and control information, may be displayed in windows on a single monitor6, or monitor6 may be switchable among the various displays. Speaker8 is also provided for eachsystem1 for output of audio information. Eachsystem1 also includes video camera5 for acquiring the video image, microphone12 for pickup of audio, usually of the teleconference participants, and keyboard10 for entry of computer data for communication to the other location, as well as for entry of commands forcomputer2 to control the teleconference transmission and receipt.
• Other input devices include an interactive[0023]graphics input device9, such as the Pen Pal Graphics™ input tablet and stylus system available from VideoTelecom Corporation, for entry of graphics information that can overlay other image information sent to theother system1B. Adocument scanner7, such as the scanning input to a conventional FAX machine, provides input of documentary information for display on theother system1B. Additional input devices includeinstrument3, such as an oscilloscope or other data acquisition device, and avideo copy stand11 which includes a video camera supported so as to view drawings or handwriting by one of the conference participants at location B. Other input and output devices may also be used in connection withsystems1.
• Referring now to FIG. 2, the construction of[0024]computer2 insystem1 will now be described in further detail. The Duran et al. paper incorporated herein by reference also provides additional detail relative tocomputer2.Computer2, as noted hereinabove, is implemented into the chassis of an IBM PC AT or equivalent personal computer. The conventional portion ofcomputer2, including the central processing unit, main memory and the like, is illustrated in FIG. 2 asCPU14. Connected toCPU14 is bus ATBUS, for communication of information betweenCPU14 and the various other units withincomputer2 according to this embodiment of the invention.
• Connected to bus ATBUS is video I/[0025]O18, which includes conventional circuitry for driving a display device such as an RGB monitor, and also includes conventional digitizing circuitry for receiving video information from a video camera and presenting the same to a computer. Accordingly, video I/O18 is coupled between the video input and output units (i.e., camera and graphics monitor, respectively) and bus ATBUS. Video I/O18 also includes a FIFO or other conventional buffer for facilitating communications to and from bus ATBUS. Also connected to bus ATBUS isdisplay adaptor19, which is a conventional adaptor (such as a VGA adaptor as is well known in the computer art), which can directly drive a display device, particularly in text mode.
• In the example of FIG. 2,[0026]interface4 receives digital communication from, and provides digital information to, various ports. During a teleconference, and especially during transmission of a message for storage as will be described hereinbelow, it is contemplated that the bulk of the transmission will be via a primary network port (shown as network port N/W in FIG. 2). This network port may be configured according to various standards; for example, the preferred network port N/W allows for selection between an RS-449 port or a pair of V.35 ports, depending upon the type of transmission desired. In addition, several user ports are also provided. These user ports include conventional RS-449 and RS-232 ports, and also a conventional FAX port. These user ports are available for parallel usage during a teleconference; for example, if one party wishes to send a FAX during a teleconference, the transmitted FAX information can be time-multiplexed into the teleconference bit stream, with the FAX port on the receiving end capable of decoding the FAX information from the multiplexed bitstream. In addition, while audio information is also preferably multiplexed into the teleconference bit stream,audio interface22 is also itself coupled tointerface4 to receive parallel telephonic communication, as well as providing communication from a microphone input, and to and from VCR and television devices.
• [0027]Audio interface22 includes one or more digital signal processor chips, such as a TMS 320C25 digital signal processor manufactured and sold by Texas Instruments Incorporated, to provide audio coding and decoding, as well as other audio information processing such as echo cancellation, an example of which is described in U.S. Pat. No. 4,965,822, assigned to VideoTelecom Corporation and incorporated herein by reference.Audio interface22 is connected directly tocommunications multiplexer20, and also to bus ATBUS.
• [0028]Communications multiplexer20 is also connected to bus ATBUS. Communications multiplexer20 is for receiving input teleconference data frominterface4, and fromaudio interface22, and routing the information to the proper location withincomputer2, and for also receiving information from its own system and presenting the same to interface4 for transmission to other systems in the teleconference. The detailed construction ofcommunications multiplexer20 will be described hereinbelow.
• For video data received at[0029]interface4,communications multiplexer20 is coupled tovideo processor24 which decodes and processes the received video information into a form suitable for display on monitors6. A preferred example ofvideo processor24 is a chip set including three i960CA RISC processors manufactured and sold by Intel Corporation, an A121 Discrete Cosine Transform circuit manufactured and sold by Inmos, and additional conventional circuitry useful in motion estimation. In this arrangement,video processor24 is coupled to bus ATBUS to receive instructions and control information fromCPU14, and is also coupled tocommunications multiplexer20 to receive video information therefrom for coding, decoding, processing and display. The output ofvideo processor24 is communicated to the video I/O18 by way of pixel bus PBUS.
• Alternatively,[0030]video processor24 may be configured as described in the Duran, et al. paper incorporated hereinabove. In this example,video processor24 includes a bank of sixteen TMS320C25 digital signal processors manufactured and sold by Texas Instruments Incorporated, each associated with 64 k bytes of conventional dual port video DRAM memory devices connected thereto, and connected together in a multiple instruction, multiple data stream (MIMD) configuration. The digital signal processors in this arrangement are programmed to perform a transform based (discrete cosine transform, or DCT) video compression algorithm.
• Further in the alternative,[0031]video processor24 may be implemented as a video compression chip set, an example of which is available from LSI Logic, for coding and decoding the video information according to the proposed CCITT video coding standard H.261. Other known and equivalent circuits and arrangements may alternatively be used asvideo processor24, for decoding and processing the video portion of the transmitted bit stream for display.
• Also connected to bus ATBUS is a[0032]conventional disk controller26, which in turn is connected tobulk disk storage28, for example having 100 Mbyte or greater storage.Disk28 will be the storage location for received messages according to this embodiment of th invention, as will be described in further detail hereinbelow.
• Referring now to FIG. 3, the construction of[0033]communications multiplexer20 according to this embodiment of the invention will now be described in further detail. Communications multiplexer20 includes acommunications CPU30, for example an i960CA RISC processor manufactured and sold by Intel, together with necessary on-chip or off-chip clock, memory, and support hardware. For example, a DMA control circuit may be provided withincommunications CPU30, for controlling DMA operations between its memory and bus ATBUS.Communications CPU30 is bidirectionally connected to bus COMBUS. Also connected to bus COMBUS isprogram memory32, for storing the routines performed bycommunications CPU30 according to this embodiment of the invention, or alternatively for storing a routine for initializingcommunications CPU30 on power-up, in which case the routines are downloaded thereto byCPU14 via bus ATBUS.
• [0034]Network port38, for example an RS-449 port or dual V.35 ports as noted hereinabove, is connected to bus COMBUS and is controlled bycommunications CPU30. It should be noted that various user ports as noted above (e.g., a FAX port, RS-232 port, and another RS-449 port) may also be connected to bus COMBUS under the control ofcommunications CPU30, for communicating information betweeninterface4 of FIG. 2 and bus COMBUS; these ports will generally not be used in the receipt of a recorded message according to the method described hereinbelow, and as such are not shown in FIG. 3 for purposes of clarity.Network port38 may include some amount of processing capability to facilitate the transmission of information between bus COMBUS andinterface4. For example,port38 may include demultiplexing circuitry so that the bitstream data on bus COMBUS (e.g., at a data rate of 112 kbps) may be applied tointerface4 in such a manner as to support transmission of two lower speed channels of information having different delay periods.
• [0035]Video FIFO40 is also connected on one side to bus COMBUS, and is connected on the other side tovideo processor24, to communicate received video data thereto for processing and display, and to receive video information therefrom for communication out of the network port. Similar buffering is provided byCPU FIFO42 connected between bus COMBUS and bus ATBUS, and byaudio FIFO44 connected between bus COMBUS andaudio interface22.
• Each of the functions ([0036]memory32,port38, and buffers40,42,44) incommunications multiplexer20 are under the control ofcommunications CPU30, in any one of several conventional ways. For example, each of the functions connected to bus COMBUS may have an address value associated therewith which is within the memory space ofcommunications CPU30; accordingly, each of these functions may be selected for communication with bus COMBUS by way of an address value placed on certain lines of the bus, or in a manner which is time-multiplexed with data on bus COMBUS. Alternatively, control lines may be connected directly betweencommunications CPU30 and each of the functions separately from bus COMBUS, such thatcommunications CPU30 selects one of the functions for connection to bus COMBUS by way of signals generated on such control lines. It is contemplated that other system techniques by whichcommunications CPU30 may control the selection and enabling of each of the functions (memory32,port38, and buffers40,42,44) withincommunications multiplexer20 will now be apparent to those of ordinary skill in the art having reference to this description.
• As described in the Duran et al. paper, the various types of data received and routed by[0037]communications multiplexer20 are in multiplexed form, suitable for communication over bus COMBUS. These communication channels include an audio channel for receiving up to 64 kbps audio information, and a video channel with a bandwidth up to that of the communications channel bandwidth (typically 112 kbps or greater) less the bandwidth portion reserved for the audio channel. Certain other channels are used on an “on demand” basis, including a 64 kbps FAX channel, an RS-232 channel (from 0.3 to 19.2 kbps), and RS-449 channel (from 0.3 to 128 kbps). In addition, bus COMBUS also carries data from bus ATBUS, on demand. Via connection of bus COMBUS through RS-449port38,communications multiplexer20 can transmit at speeds from 9600 bps to 768 kbps; conventional systems such as the CS420R system noted hereinabove are typically optimized at particular frequencies, for example 384 kbps, or one-fourth the bandwidth of a T1 communications line.
• The transmitted message according to this embodiment of the invention includes data corresponding to multiple input/output media, as noted hereinabove. Depending upon the available bandwidth and the multiplexing arrangement, various conventional coding and compression schemes may be used for data of each type. For example, the data corresponding to the motion video portion of the message may be compressed according to CCITT recommendation H.261, which is a lossy compression scheme. Audio information, especially at lower data rates, will generally also be compressed, although the compression scheme will differ from that for motion video. Other data corresponding to other media may be coded as best suitable for the application, or may not be coded or compressed at all.[0038]
• Referring now to FIG. 4, the operation of[0039]systems1, includingCPU14 andcommunications CPU30 programmed according to this embodiment of the invention, in performing the function of transmission of a multi-media message and its storage for later retrieval, will now be described in detail. FIG. 4 includes a pair of flow charts, one for the sending system and one for the receiving system. It is contemplated that one of ordinary skill in the art, having reference to this specification, will be able to readily programCPU14 andcommunications CPU30 to perform the functions and operations described herein.
• The communications network used in connection with this embodiment may be a switched (i.e., dial-up) network, a dedicated network, or another type of network such as a local-area-network (LAN). In the example of a switched network, the transaction begins with[0040]process50, where the sending operator causes the sending system to initiate a telephone call over the digital communications network in, the conventional manner. Part of the call initiation process may include selection of the data rate of the desired transmission and communication of this data rate to the receiving unit; alternatively, the data rate may be defined by the network. It should be noted that the CS420R system noted hereinabove, when used with a switched network, provides for user-friendly call initiation, including menu-driven destination selection and call placement. Upon receipt of the call, the receiving system detects a ring inprocess60 and determines, indecision62, whether or not a person on the receiving side has actively answered the call; if so, the normal call mode is entered, and the system functions in the conventional manner to perform a two-way or multiple-way conference, inprocess64.
• In the event no person actively answers the call, however, an automatic answer function will be performed on the receiving end (if enabled). In the automatic answer mode, if no person has answered the call after a selected number of rings, the receiving system will enter a record mode, in[0041]process66, and will indicate to the sending unit that such a mode has been entered.
• The indication of record mode provided by the receiving system will cause the sending system to exit a waiting loop. In the waiting loop, sending system determines in[0042]decision52 if the call has been answered. If not, the sending system determines indecision54 if the sending operator has hung up, exiting the call if such is the case. If the sending operator has not hung up, the waiting loop continues withdecision52. Upon receipt of the indication from the receiving unit that the call has been answered, the sending unit determines, indecision56, if the receiver is in record mode or normal mode. Normal mode calls are processed in the conventional manner, shown asprocess58.
• The receiving system, once in record mode, has meanwhile interrogated its disk storage[0043]28 (or other storage unit in which received messages are to be stored) to determine the length of message available for storage (process68). The calculation of the maximum length message is preferably performed by dividing the space available indisk storage28 by the data rate desired by the sending system to arrive at a maximum time duration which is available for the transmitted message. It should be noted that various other criteria may be used to determine the maximum message length, including a preselected limitation on the length of a particular message for all sender, or for particular senders. The calculated maximum message length is communicated to the sending system for receipt inprocess70, and the receiving system awaits the reply.
• At this point, the sending operator can decide whether or not to transmit a message based on the maximum message length information communicated by the receiving system, which is preferably displayed on one of the monitors[0044]6 of the system. Other information may also be transmitted by the receiving system at this time, including identity information (e.g., “You have reached . . . ”) and a timing message (e.g., “Please begin your message at the tone.”). Alternatively, the receiving system may begin recording instantaneously with the initiation of the call.Decision72 determines whether or not the sending operator has decided to hang up, in which case the process exits. If the sending operator has not hung up,process74 transmits a record signal to the receiving system, indicating that a message is about to be transmitted for recording by the receiving system.Decision80 of the receiving system will then pass control to the recording process, rather than exiting as would be the case if no record signal were transmitted by the sending system. The receiving system will then enterprocess82, to await and process the message.
• The sending system will then receive information from its various input devices, such as microphone[0045]12, cameras5, scanner orFAX input7,graphics tablet9, keyboard10, and other inputs, including information processed byCPU14 incomputer2 itself. Each of the input devices useful in a conventional video teleconference, as performed by the CS420R system noted hereinabove, may be transmitted by the sending system. Inprocess76, the sending system will send the combined information over network DCN in the conventional manner, as though a normal two-way multi-media conference were taking place. During this time, additional features such as display of the remaining message time available from the maximum message length sent by the receiving unit, may be incorporated. At such time as the message is complete, as determined by the sending operator indecision78, the sending operator will merely hang up.
• Referring now to FIG. 5, in conjunction with FIGS. 6[0046]aand6b, the operation ofprocess82 in the receiving system, programmed according to this embodiment of the invention, by which transmission data from the sending system are received and stored for later retrieval, will now be described in detail. Referring first to FIG. 6ain combination with FIG. 5,process82 begins withprocess90, in whichcommunications CPU30 enablesnetwork port38 to receive data frominterface4 and, in turn, from the sending system. Inprocess92,communications CPU30 grants bus COMBUS access tonetwork port38, so that the data received thereby are placed on bus COMBUS.Communications CPU30 receives the data from bus COMBUS, and stores the data in its memory, inprocess94. The data flow frominterface4 tocommunications CPU30 is illustrated in FIG. 6a.
• The data received by[0047]communications CPU30 inprocesses90,92,94 may correspond to a single sample in time, or may correspond to a series of time samples, depending upon the memory capacity of communications CPU30 (either on-chip or off-chip), the processing to be performed bycommunications CPU30, and other factors. Accordingly, the flow of data illustrated in FIG. 6amay correspond to varying times of transmission. Regardless of the duration of the storage,communications CPU30 processes the data stored inprocess94, and reformats it into a conventional form suitable for storage indisk storage28 ofcomputer2, as illustrated in FIG. 2. Certain information in addition to the transmitted data may be stored in conjunction with the data, an example of such additional information including the data rate at which the information was received. Reformatting may not be necessary in some applications, in which case process96 (and, perhaps, process94) is not applicable.
• Once the data have been received by[0048]communications CPU30 and stored withincommunications multiplexer20, the storage of the received data in the memory ofcomputer2, preferably fixed non-volatile storage such asdisk storage28 of FIG. 2, can now be performed. Alternatively todisk storage28, the message may be stored in other computer-accessible memory, such as random access memory incomputer2, or disk or RAM located in another computer or network and which is accessible by computer2 (e.g., memory in a LAN server).
• In this preferred embodiment, it should be noted that[0049]CPU14 incomputer2 has preferably pre-assigned a location indisk storage28 for the data to be received in the transmission, according to the usual steps and protocol for the operating system resident incomputer2. Referring now to FIG. 6bin combination with FIG. 5,process98 is performed in whichcommunications CPU30polls CPU FIFO42 to determine its availability to receive data from bus COMBUS. Upon determining thatCPU FIFO42 can receive such data,communications CPU30 then writesCPU FIFO42 with a portion of its memory contents corresponding to the reformatted received data, for eventual transfer to bus ATBUS. Alternatively, the transfer of data may be performed by way of direct memory access (DMA), or by other program controlled I/O operations. The flow of data incommunications multiplexer30 during this portion of the operation is illustrated in FIG. 6b.
• Referring back to FIG. 4, upon[0050]communications CPU30 writing data intoCPU FIFO42,CPU14 incomputer2 will have interrogateddisk controller26 to ensure that sufficient space remains available indisk storage28 for storage of the new data (decision84 in FIG. 4). If disk space is no longer available, the operation is terminated. If sufficient disk space remains,process86 is performed by whichCPU14polls CPU FIFO42 to determine if it contains data to be read byCPU14, and enablesCPU FIFO42 to begin placing this data onto bus ATBUS.CPU14 furthercontrols disk controller26 to allow the data on bus ATBUS to be stored indisk storage28, in association with a conventional file specification or other designation to facilitate its subsequent retrieval. The state of the line is again interrogated in decision88 (either directly, or by determining whether or not data has been transmitted within a time-out period) to determine if the sending unit has hung up (in which case the process terminates), withprocess82 repeating if the transmission is continuing. The loop ofprocess82, in which the received data are processed and stored, thus either continues until the available space indisk storage28 is filled, or until completion of the multi-media message.
• Referring now to FIG. 7, the process of playback of a previously stored message will now be described. It is first contemplated that, upon completion of the transmitted message, the receiving system will indicate to its owner or user that a new message has been received and awaits review by the operator. Such indication may be made by way of displaying this status on a monitor or other visual indicator, or by other techniques such as an automatic phone call to the system manager. It is contemplated that a menu-driven system may be readily provided within the receiving system, such that the operator may view a menu of stored messages, and select one or more desired received messages for playback, illustrated as[0051]process102 in FIG. 7.
• Upon selection of a desired message for playback,[0052]CPU14 will execute the necessary operating system instructions to access the associated memory locations indisk storage28 of the data for the desired message. It should be noted that the retrieval and processing of the stored message must be done at sufficient speed that the system will display the message in real time. It is therefore preferred that the retrieval of data fromdisk storage28 be buffered in a conventional manner (e.g., by way of a circular buffer, or double buffered as will be described hereinbelow), so thatdisk storage28 may retrieve a number of records while other records are being transferred tocommunications multiplexer20. It should be noted that the storage of the received message described hereinabove relative to FIGS. 5, 6aand6bmay also be performed in such a buffered manner, if necessary.
• The example of retrieval of a stored message by way of a double buffered system will now be described in detail. It is contemplated that the selection of the buffering scheme will depend upon the data rate, the amount of buffer memory available, and the particular application. In the double buffered case,[0053]process105 sets the state of a buffer pointer B to thevalue 1, and a record counter N to thevalue 1, prior to commencing retrieval of the data fromdisk storage28. As will be evident hereinbelow, the number of records associated with a message will be stored byCPU14 so that the end of the retrieval process can be determined. Inprocess106, the record N is retrieved fromdisk storage28, and stored in buffer B in the memory ofCPU14, according to the values of the buffer pointer and record counter. Each record N may consist of a series of time samples at the stored data rate, and as such corresponds to a period of time in the stored message. The size of a typical record according to this embodiment of the invention is contemplated to be on the order of 4 kbytes. Inprocess108, once the pointed-to buffer B receives record N,process111 of FIG. 8 is initiated bycommunications multiplexer20 in conjunction withCPU14, in which the contents of record N are displayed by the receiving system for viewing by the recipient of the message.
• Referring now to FIG. 8, in combination with FIG. 9[0054]a, the operation ofcommunications multiplexer20 in the receiving system will now be described during the playback mode. In parallel withprocess111 of FIG. 8,CPU14 writes the data to be transferred intoCPU FIFO42 by way of bus ATBUS, after pollingCPU FIFO42 for its availability in the conventional manner. It is preferred that the priority of the playback operation be quite high, so that the real-time effect of the retrieved message is maintained.
• [0055]Communications CPU30 reads the contents ofCPU FIFO42 inprocess116, during whichCPU FIFO42 places the received data onto bus COMBUS for storage in the memory ofcommunications CPU30. The data flow for this operation is illustrated in FIG. 9a. After receipt of the data from bus COMBUS,communications CPU30 reformats the data into a form suitable for decoding, processing and output by the various functions (process118).
• It should be noted that the transfer of data from[0056]disk storage28 back intocommunications CPU30 for playback may alternatively be performed by way of direct memory access (DMA). Further in the alternative, other program controlled I/O operations may be used to transfer the data from bus ATBUS tocommunications CPU30.
• The reformatted data in the memory of[0057]communications CPU30 are then placed onto bus COMBUS in time-multiplexed form, in combination with the enabling of the various functions connected to bus COMBUS to receive the reformatted data and communicate the same to their respective destinations (process120). The time-multiplexing performed bycommunications CPU30 is identical to that performed for real-time message information being received from a sending unit; as a result, the information forwarded to the video; audio, and CPU portions of the receiving system is presented as though the message were incoming at that time.Process122 indicates the time-multiplexing of data on bus COMBUS, for receipt byvideo buffer40,audio buffer44, andCPU buffer42. Decoding and processing of the video data byvideo processor24,audio interface22, andCPU14, is then performed in the same manner as during receipt of live teleconferencing data. The most direct data flow for this operation is illustrated in FIG. 9bfor clarity; alternative paths, such as sending the data to networkport38 and its replacement onto bus COMBUS, may be preferred depending upon the timing requirements of the particular system and for ease of programming. Upon completion of the time-multiplexed output ofprocess122,communications CPU30 thenpolls CPU FIFO42 to determine if new data are present therein, and reads such data, if such is the case, inprocess116, for playback of the next record.
• Referring back to FIG. 7, during such time as the output of data by[0058]communications multiplexer20 is occurring,CPU14 compares the value of record N against the terminal value to determine if additional records are to be transmitted tocommunications multiplexer30; if not, the process terminates. If additional records remain, the record counter is incremented, and the other buffer (i.e., not the buffer from which DMA is currently taking place) is selected.Process106 is then performed, in which the next record is retrieved from disk storage into the buffer, followed byprocess108 at such time asCPU FIFO42 is available to receive the next record.
• It is contemplated that transmission errors may occur during the transmission of a message for storage, as in the case of live teleconferences. Correction of the data stream for such errors is useful in live teleconferences, but is not critical, as the receiving party is able to ask the sending party to repeat himself or herself, or to retransmit the data, if the errors are of sufficient extent to affect the meaning of the communication. In the above-described process of transmission and storage of a message for later retrieval, however, the recipient is not available to ask for retransmission or repetition of the message. Accordingly, it is preferred that some type of on-line error correction be utilized by the receiving system.[0059]
• Various approaches to error correction are contemplated for use with the present invention. A first approach would be to apply conventional forward error correction techniques to the incoming bit stream, either by way of[0060]communications CPU30 in reformattingprocess96 described hereinabove, or by special network interfac circuitry. Such forward error correction, as is well known, is not sufficient to correct all errors of a particularly serious nature.
• A second approach to error correction contemplated for use in the present invention is the use of an error checking protocol, such as the well-known HDLC protocol, which has the capability of detecting corrupted data packets, and requesting the sending system to retransmit the bad packets. According to this approach,[0061]communications multiplexer20 would transfer only non-corrupted data packets to bus ATBUS for storage indisk storage28, so that no corrupted data packets would be received bycommunications multiplexer20 during playback; data packets would then be retransmitted similarly as in a conventional interactive video teleconference which employs such a retransmission protocol.
• Of course, transmission errors may merely be ignored. Errors in video and audio transmission, while causing temporary disruption of the played-back message, will generally not be fatal to the system as the video and audio compression algorithms will eventually recover after the error. It should be noted, however, that data transfers, such as spreadsheet or graphics information, may be received and stored in a corrupted fashion if errors ar merely ignored.[0062]
• It is further contemplated that additional interaction between the sending and receiving systems may be provided to enhance the quality of the stored message. For example, analogously to conventional voice mail systems, some amount of editing capability may be provided in the receiving system at the initiation and control of the sending operator. Such editing may allow for playback of the stored message at any time during its transmission, together with the ability to re-record some or all of the message transmitted at that time. Initiation of such editing may be provided by transmission of a particular command by the sending system which is received, decoded and executed by the receiving system; alternatively, an edit function may be provided upon completion of the transmission.[0063]
• Further in the alternative, it is contemplated that the present invention may be practiced by way of transmission of a multiple-media message from a sending teleconference system directly to computer storage, rather than to a receiving teleconference system as described hereinabove. The computer storage may be located at the sending, system and accessible by a receiving teleconference system, located at the receiving system, or at another location accessible by both. The sending system may then communicate the existence and location (address, file specification, or the equivalent) of the stored message, by way of a conventional phone call, letter, telex, FAX, or even a brief multiple medium message as described hereinabove, to the receiving system. The receiving system may “playback” the message at a later time by accessing the appropriate storage unit and receiving the stored bitstream through its network port as though a real-time incoming message were being received, with display of the multiple medium message occurring in the same manner. The receiving system may alternatively download the stored bitstream data into its own storage, for playback at a later time through its[0064]communications multiplexer20 in the manner described hereinabove relative to FIGS. 7, 8,9aand9b.
• It should further be noted that while a two-party transmission is described hereinabove, the present invention is also applicable to multiple-party message storage and retrieval. For example, the sending operator may record a message in the sending system itself, in similar manner as the receiving system receives and stores a message; at the desired transmission time, the data stream may be transmitted to one or multiple systems in the format in which it was stored in disk, without requiring the[0065]communications multiplexers20 in the receiving systems to reformat the transmitted message.
• Alternatively, or in addition, the sending system may utilize a system, such as a Multipoint Control Unit, which is capable of dialing several locations at once, and communicating the message to each for storage thereat. This allows for a broadcast multiple medium message to be transmitted for later retrieval at each receiving location, according to the recipients' particular schedules and needs.[0066]
• The above-described system and method thus provides the capability for the transmission of a multiple medium message for storage and later retrieval and playback. The ability of parties to fully communicate their desired messages using the most appropriate media is thus improved, as scheduling conflicts are avoided. Furthermore, the recipient of the message is able to view the message at his convenience, and in the order desired (if multiple messages have been stored). The communications between the sending and receiving unit, particularly relative to the maximum message length, further enhances the ability for meaningful messages to be transmitted.[0067]
• While the invention has been described herein relative to its preferred embodiment, it is of course contemplated that modifications of, and alternatives to, this embodiment, such modifications and alternatives obtaining the advantages and benefits of this invention, will be apparent to those of ordinary skill in the art having reference to this specification and its drawings. It is contemplated that such modifications and alternatives are within the scope of this invention as subsequently claimed herein.[0068]

Claims (30)

We claim:

1. A method of receiving a multiple medium message, comprising:

receiving message data from a communications network;

storing the received message data in memory;

retrieving the stored message data; and

communicating the retrieved stored message data to a plurality of output devices.

2. The method ofclaim 1, wherein the plurality of output devices comprise:

a video display; and

an audio output.

3. The method ofclaim 1, further comprising:

decoding at least a portion of the retrieved stored message for output.

4. The method ofclaim 3, wherein one of said plurality of output devices comprises:

a video processor;

and wherein said decoding step is performed by said video processor.

5. The method ofclaim 4, wherein said decoding step is performed by said output devices after said communicating step.

6. The method ofclaim 1, further comprising:

formatting the received message data, prior to said storing step.

7. The method ofclaim 1, wherein said receiving step comprises:

enabling an interface to the telecommunications network to place the message data on a local bus;

placing the message data on the local bus into a local memory; and

communicating the message data from the local memory to a computer bus.

8. The method ofclaim 7, further comprising:

formatting the message data after said placing step; and

replacing the formatted message data into said local memory.

9. The method ofclaim 1, wherein said retrieving step places said stored message onto a computer bus;

and wherein said communicating step communicates the data from said computer bus to said plurality of output devices.

10. The method ofclaim 9, wherein said communicating step comprises:

communicating the retrieved message data from the computer bus onto a local bus; and

enabling a plurality of output devices coupled to the local bus to receive the retrieved message data.

11. The method ofclaim 10, wherein said enabling step is performed in time-multiplexed fashion.

12. The method ofclaim 10, wherein said receiving step comprises:

enabling an interface to the telecommunications network to place the message data on the local bus;

placing the message data on the local bus into a local memory; and

communicating the message data from the local memory to a computer bus.

13. The method ofclaim 12, further comprising:

formatting the message data after said placing step; and

replacing the formatted message data into said local memory.

14. The method ofclaim 1, further comprising:

performing error correction on said received message data.

15. The method ofclaim 1, further comprising:

prior to receiving said message data, calculating the maximum message length to be stored; and

responsive to receiving a call initiation signal from a remote location, communicating the maximum message length to said remote location.

16. A system for storing a transmitted multiple medium message for later retrieval and playback, comprising:

an interface for connection to a telecommunications network;

a plurality of output devices for displaying a message in a plurality of media;

a communications multiplexer coupled to said interface, coupled to said plurality of output devices, and coupled to a computer bus; and

a memory, coupled to said computer bus;

wherein said communications multiplexer is operable in a store mode to receive information from said interface and communicate the information to said computer bus for storage in said memory, and is operable in a playback mode to receive information from said computer bus and communicate the information to said plurality of output devices.

17. The system ofclaim 16, wherein said plurality of output devices comprise:

a video display device; and

an audio speaker.

18. The system ofclaim 16, wherein said communications multiplexer comprises:

a local bus;

a CPU coupled to said local bus;

a telecommunications port coupled to said interface and to said local bus; and

a buffer coupled between said local bus and said computer bus;

wherein, said CPU is programmed to control access of said telecommunications port and said buffer to said local bus in said store mode.

19. The system ofclaim 18, wherein said communications multiplexer further comprises:

a plurality of output buffers coupled to said local bus;

wherein said CPU is programmed to control access of said plurality of output buffers to said local bus in said playback mode.

20. The system ofclaim 19, wherein said CPU comprises:

a processing unit; and

a local memory.

21. The system ofclaim 20, wherein said CPU is also programmed to receive data from said local bus and store the received data in its local memory.

22. The system ofclaim 19, wherein said CPU is also programmed to process data received from said local bus.

23. The system ofclaim 18, wherein said CPU is also programmed to control said buffer to effect direct memory access between said local bus and said computer bus.

24. A method of receiving a multiple medium message for later display, comprising:

receiving message data from a communications network, said message data including a plurality of portions corresponding to different output formats;

storing the received message data in memory;

retrieving the stored message data; and

presenting the retrieved stored message data for receipt by humans.

25. The method ofclaim 24, wherein the presenting step comprises:

communicating the retrieved stored message data to a visual display.

26. The method ofclaim 25, wherein the communicating step comprises:

communicating the retrieved stored message data to a video processor;

decoding the retrieved stored message into displayable form; and

displaying the decoded message on a visual display.

27. The method ofclaim 24, wherein the presenting step comprises communicating the retrieved stored message to a plurality of output devices.

28. The method ofclaim 27, wherein the plurality of output devices comprise:

a video display; and

an audio output.

29. The method ofclaim 27, wherein said retrieving step places said stored message onto a computer bus;

and wherein said communicating step communicates the data from said computer bus to said plurality of output devices.

30. The method ofclaim 24, further comprising:

prior to receiving said message data, calculating the maximum message length to be stored; and

responsive to receiving a call initiation signal from a remote location, communicating the maximum message length to said remote location.

Images