US20170244835A1 - Computer-Implemented System and Method For Facilitating Call Sessions Via Messages - Google Patents

Abstract

A computer-implemented system and method for facilitating call sessions via messages is provided. Two or more call sessions are monitored and each call session is processed by receiving speech utterances from one of the participants and transcribing the speech utterances into text. The transcribed speech utterances are provided to another participant in the call session. Text messages from the other participant are received in response to the transcribed speech utterances and provided to the participant as synthesized speech. During each call session, an ongoing log of messages including the transcribed speech utterances and the text messages for that call session is generated. Annotations from one or more of the participants to that call session are added to the call log.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This non-provisional patent application is a continuation of U.S. patent application Ser. No. 15/043,490, filed Feb. 12, 2016, pending; which is a continuation of U.S. Pat. No. 9,264,545, issued Feb. 16, 2016, which is a continuation of U.S. Pat. No. 8,666,032, issued Mar. 4, 2014; which is a continuation of U.S. Pat. No. 8,170,197, issued May 1, 2012; which is a continuation-in-part of U.S. Pat. No. 7,292,689, issued Nov. 6, 2007; which claims priority to U.S. Provisional Patent Application Ser. No. 60/364,555, filed Mar. 15, 2002 and to U.S. Provisional Patent Application Ser. No. 60/403,354, filed Aug. 13, 2002, the priority dates of which are claimed and the disclosures of which are incorporated by reference.
COPYRIGHT NOTICE
• A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
FIELD
• The present invention relates in general to automated call center operation and, in particular, to a computer-implemented system and method for facilitating call sessions via messages.
BACKGROUND
• Customer call centers, or simply, “call centers,” are usually the first direct point of contact for customers seeking direct assistance from manufacturers and service vendors. Call centers are reachable by telephone and provide a single source for customer support and problem resolution. Although World Wide Web-based customer support is becoming increasingly available via the Internet, call centers still offer a convenient and universally-available forum for remote customer assistance.
• As customer satisfaction and good will depend significantly on service after the sale, vendors spend substantial time, money and effort in ensuring effective call center operation. Customer assistance professionals are trained in providing both effective and courteous communication and informed and accurate product and service information. Nevertheless, the volume of call traffic can often exceed the capabilities of human customer assistance agents, and a range of automated call center systems are presently used to help bridge the gap between the need to provide responsive assistance and the limits of human call center staff.
• Typically, in existing automated call center systems, customers are put on hold until an agent is available to take their call. While on hold, an automated system typically collects information from the customer, such as account number, to determine a priority of service. Such a system may also provide automated menus that attempt to classify the call into basic transaction types, for instance, based on language spoken.
• When an agent is available to take the call, the agent will greet the customer, may ask for identification information, and will attempt to determine the nature of the call, often by asking for the same information that the customer previously provided to the automated system. The agent then takes some action, such as performing troubleshooting and providing the caller with instructions for resolving the problem. Further, to assist in improving customer support, the agent will usually log the customer information provided by the caller.
• Each customer call is typically an interactive process. During the interaction, the agent may put the customer on hold while the agent gathers information, takes notes, or sometimes handles other customers. Finally, after call completion, the agent will summarize the call as a call log entry. The interactive process is repeated for each new caller throughout the course of the day. This interactive process is time inefficient. Agents are forced to wait for customers to complete their instructions while customers similarly remain on hold while agents are assisting other callers, researching a problem resolution, or creating call logs.
• A customer interaction system is described in U.S. Pat. No. 6,778,660 to Fromm, issued on Aug. 17, 2004, the disclosure of which is incorporated by reference. The system enables agents to simultaneously process voice contacts from telephonic callers by storing incoming voice signals for time-shiftable and fast playback. Calls are routed to assigned agents when possible. If an agent is busy, the user is asked to state his question for storage in one or more queues. An agent subsequently processes and responds to the question in person, in a manner similar to that utilized in Web chat interactions. Agents respond to voice recordings in the order in which received with additionally provided fast playback that enables an agent to catch up with recorded messages. However, both user and agent messages remain as spoken speech recorded in an audible, non-textual format and accordingly require the full attention of the assigned agent.
• Call centers and, in particular, agents, are presented with a related set of problems following the completion of a call from a customer. Incoming calls that originate with callers often need to be reviewed for quality control and to identify common problems, keywords, and patterns that can assist the call center in staying current with customer concerns and in generating post-call statistics. Conversely, outgoing calls that originate with a call center are faced with additional concerns where the call is answered by some form of automated response, such as an answering machine or user call menu.
• Efficiently processing calls can be a time-consuming and tedious task, especially where the calls are lengthy or include extended periods of inactivity, for instance, where the caller is placed on hold. Similarly, consistently categorizing the participants in a call can be involved and may require identifying roles, genders, languages, and other characteristics of the call participants. Finally, only a part of a call may be of interest while the remainder can be disregarded. Repeat calls to the same type of callee can often be streamlined, for instance, by generating a call “macro” that navigates through an answering machine or user call menu automatically.
• Accordingly, there is a need for an approach to providing automated call center operation that allows highly responsive caller support with a minimum of agent idle time and caller hold time.
• There is a further need for an approach to providing efficient caller message processing using transcribed and synthesized speech utterances as an internal medium of communication within the automated call center.
• There is a further need for an approach to providing analysis of completed calls, as well as in-progress calls, whether incoming or outgoing relative to a call center. Preferably, such an approach would enable automated processing of calls and identification of participant characteristics.
• There is a further need for an approach to providing tools to assist with creating automatic call-outs that streamline the calling process in situations where the pattern of the call is generally known beforehand.
SUMMARY
• In the described embodiment, an agent and customer communicate through voice messages using a digitized voice-driven system. From the customer perspective, the experience appears to be an interaction with an intelligent machine. The interaction is similar to calling a legacy automated call center system staffed with human agents, but the customers are aware that the agent is automated, not human.
• Preferably, the system voice is clear and human-like, but is recognizable as a machine voice. Slight delays in responses can occur compared to speaking with a human agent, although the customer does not have to repeat information already provided and is generally not put on hold. Upon request, the system can repeat the information provided to the customer, and consistently appears to be patient and polite.
• Operationally, the system differs from legacy systems. Instead of providing full-time voice-to-voice communications, the system gives agents the ability to control a continuum of increasingly automated responses in the form of a “sliding” control. For most interactions, every customer speech utterance is recorded and stored, is digitally transcribed into a text message and is presented to an off-line agent through a visual interface on a workstation. The agent can read or optionally listen to each utterance upon arrival, as well as to previous utterances. As well, the agent can annotate or manually re-transcribe each of the customer utterances as needed.
• Once a session has been established, each human agent can communicate indirectly with customers by typing written responses at their workstation. Each written response is converted into speech following completion and is played to the customer.
• The agent can also choose pre-formed responses, thereby saving time and communicating in a consistent, uniform manner. The pre-formed responses can include an associated form containing parameterized variable fields that are completed by the agent or by the system to fill in, for example, dates or names. The completed pre-formed response is converted into speech. Alternatively, pre-formed responses can be pre-recorded as sound bites and staged in an audio cache for immediate playback.
• In addition to indirect communication via their workstation, each human agent can also accept live calls from customers in a real time, voice-to-voice mode. Live call processing may be required to handle crises or other service needs that are not amenable to automation, or to provide those services to customers, which are not generally acceptable when provided via automation.
• Furthermore, when an agent identifies a call that matches a recognized problem or frequently asked question, the agent can choose a predefined “script” to prompt and collect or simply provide the customer with information in a step-by-step manner. For example, a script could be used to collect a customer's personal information for a credit application, or to provide instructions on using a feature of a consumer product in a customer support application. Thus, the ability of an agent to interact with customers through manually or automated text responses converted into speech or through pre-recorded or live voice responses provides a flexible and sliding level of agent control adaptable to a wide range of customer service situations.
• The system also provides an automatic journaling function. By the time each call ends, the system will have collected a complete and fully transcribed log of the conversation. The human agent need not manually transcribe a log entry, as the information contained in the system-generated log is already in a format that can be stored in a database and can be easily mined for data. Manual annotations can be added to the log entry, as needed.
• In a further embodiment, the system provides post-mortem call analysis that processes both incoming and outgoing calls that have completed or, in a still further embodiment, are in-progress. Verbal speech utterances are identified and processed, which can include presenting the identified speech utterances to an agent and executing commands on the speech utterances. Processing can optionally include identifying speaker characteristics of the speech utterances and marking select speech utterances after first performing speech recognition.
• A still further embodiment provides a computer-implemented system and method for facilitating call sessions via messages. Two or more call sessions are monitored and each call session is processed by receiving speech utterances from one of the participants and transcribing the speech utterances into text. The transcribed speech utterances are provided to another participant in the call session. Text messages from the other participant are received in response to the transcribed speech utterances and provided to the participant as synthesized speech. During each call session, an ongoing log of messages including the transcribed speech utterances and the text messages for that call session is generated. Annotations from one or more of the participants to that call session are added to the call log.
• Empirically, from an agent perspective, an average customer service call lasts seven minutes when using a legacy call center, of which two minutes are spent on an introduction and setup and an additional two minutes on wrap-up and documentation. The described embodiment eliminates most of the time spent on setup and wrap-up and reduces the time an agent spends on a call by about 30%. From a customer point of view, although the overall length of the call may be the same, hold times are eliminated while slight yet unobjectionably longer delays between agent's responses may occur.
• With the extensive logging and visual presentation to the agent, the agent can keep track of more than one session with a customer. An agent may handle multiple calls simultaneously, while each customer will hardly notice any degradation in call quality. In the described embodiment, an agent can handle four calls simultaneously. However, an arbitrarily large number of simultaneous calls could be maintained by the system for each agent and would only be subject to physical computational constraints, such as available processing power and memory. Each agent is able to handle multiple calls by not having to listen to each customer as they speak. Instead, agents can listen to or read transcribed customer utterances, thereby each allowing their attention to be split among multiple calls. The ability to use pre-recorded forms to gather or provide standard information further increases an agent's ability to handle multiple calls. For example, while a customer is interacting with a script collecting personal information, such as first and last name, social security number, address and phone number, and so forth, the agent can handle other callers.
• An embodiment is a system and method for providing a message-based communications infrastructure for automated call center operation is described. A call from a user into a call center is accepted. The accepted call includes a stream of transcribed verbal speech utterances. Each transcribed verbal speech utterance is recorded as a user message. The accepted call is assigned to a session, which is then assigned to an agent. The call is progressively processed in the assigned session by presenting each user message to the assigned agent, executing commands responsive to the assigned agent, and sending an agent message to the user. The agent message includes a stream of synthesized audible speech utterances.
• A further embodiment is a system and method for efficiently operating an automated call center through text-based messaging. A voice-based caller is interfaced through a telephonic medium. Audible speech utterances are exchanged during a call into an automated call center. Each call is managed as a session and each session is assigned to an agent. Incoming audible speech utterances are converted into machine-processable text messages provided to the agent assigned to the session. An action specified by the assigned agent is executed. Outgoing machine-processable text messages are processed into audible speech utterances provided to the voice-based caller transacting the session.
• A further embodiment provides a system and method for message-based call communication. Speech utterances are obtained from a caller during a call with an agent. Each of the speech utterances are transcribed into a text message and the text messages are provided to the agent. One or more written responses, in reply to each speech utterance from the caller, are received from the agent. The written responses are converted to synthesized speech for providing to the caller. The speech utterances from the caller and the synthesized speech from the agent are processed and one or more characteristics of the caller and the agent are identified. A role for each of the caller and the agent is determined based on the identified characteristics.
• Still other embodiments of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein is described embodiments of the invention by way of illustrating the best mode contemplated for carrying out the invention. As will be realized, the invention is capable of other and different embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and the scope of the present invention. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a functional block diagram showing an automated call center operational environment.
• FIG. 2 is a process flow diagram showing, by way of example, a typical user call sequence, as transacted in the automated call center operational environment ofFIG. 1.
• FIG. 3A is a block diagram showing a system for providing a message-based communications infrastructure for automated call center operation, in accordance with the present invention.
• FIG. 3B is a block diagram showing a system for providing a message-based communications infrastructure for automated call center operation, in accordance with a further embodiment the present invention.
• FIG. 4 is a process flow diagram showing variable automation levels provided using the system ofFIG. 3A.
• FIG. 5 is a block diagram showing a schema for storing records in the logging database of the system ofFIG. 3A.
• FIG. 6 is a process flow diagram showing, by way of example, the control flow followed in processing a script using the system ofFIG. 3A.
• FIG. 7 is a screen shot showing, by way of example, a set of call center service windows generated by the system ofFIG. 1.
• FIG. 8 is a flow diagram showing a method for providing a message-based communications infrastructure for automated call center operation, in accordance with the present invention.
• FIG. 9 is a flow diagram showing a routine for assigning a session for use in the method ofFIG. 8.
• FIG. 10 is a flow diagram showing a routine for processing a session for use in the method ofFIG. 8.
• FIG. 11 is a flow diagram showing a routine for processing a session by an agent for use in the routine ofFIG. 10.
• FIG. 12 is a flow diagram showing a routine for processing a session by a script engine for use in the routine ofFIG. 11.
• FIG. 13 is a flow diagram showing a routine for executing a script for use in the routine ofFIG. 12.
• FIGS. 14A and 14B are process flow diagrams showing, by way of example, a typical post-call sequence, as transacted in the automated call center operational environment ofFIG. 1.
• FIG. 15 is a block diagram showing a system for providing a message-based communications infrastructure for automated call center post-call processing, in accordance with the present invention.
• FIG. 16 is a flow diagram showing a method for providing a message-based communications infrastructure for automated call center post-call processing, in accordance with the present invention.
• FIG. 17 is a flow diagram showing a routine for processing a call for use in the method ofFIG. 16.
• APPENDIX A provides a sample grammar for use in the described embodiment.
DETAILED DESCRIPTIONSystem for Providing a Message-Based Communications Infrastructure
• FIG. 1 is a functional block diagram showing an automated call centeroperational environment10. By way of example, a multiplicity of users call into anautomated call center11, preferably through telephonic means. The telephonic means include Plain Old Telephone Service (POTS)12, cellular andsatellite telephones13, Internet telephony (IPTel)14, Voice over IP (VoIP)15, and other forms of telephony and voice-based communications, as would be recognized by one skilled in the art. Users could also call or interface to theautomated call center11 through data transmission means, such as aninternetwork17, including the Internet.
• Independent of call origination, each user call is routed through a telephone company (Telco)public interchange16 or equivalent call center to theautomated call center11. Although shown with reference to a Telcopublic interchange16, any other form of telephonic or equivalent call networking system transmitting voice or data signals over various signal carrier mediums, including conventional land lines; radio, satellite or other forms of signal carriers; light wave or sound wave exchange systems; or equivalents thereof, could also be utilized, as would be recognized by one skilled in the art.
• Theautomated call center11 provides a single source for support and problem resolution for customers seeking direct assistance from manufacturers and service vendors, althoughautomated call centers11 can also be used in other areas of commerce, as would be recognized by one skilled in the art. The terms “user” and “customer” are used interchangeably herein and both refer to a caller to theautomated call center11. Although theautomated call center11 is shown as a single point within the automated callcenter operation environment10, theautomated call center11 could consist of one or more logically interconnected but physically separate, including geographically removed, operations, which provide a logically unified automated call center, as would be recognized by one skilled in the art.
• FIG. 2 is a process flow diagram showing, by way of example, a typicaluser call sequence20, as transacted in the automated call centeroperational environment10 ofFIG. 1. Upon calling into theautomated call center11, each user receives an initial greeting and an informational message providing a synopsis of caller options. The caller options enable the user to navigate through to a specific topic area for assistance or support. Following application, the user engages in acustomer support scenario22 with an agent, which is either a live person or an automated prompt, such as with an automated voice response system, to enable information collection and problem troubleshooting. Note that thescenario22 can be delayed by agent unavailability, caller volume capacity limits, and other factors that can delay customer response. As required,service provisioning23 is provided to the user, either directly in the course of the call or indirectly through a service request dispatch. Finally, the call ends in a wrap-up24, which provides closure to the call and a departing salutation. Other forms and variations of customer call sequences are feasible, as would be recognized by one skilled in the art. Importantly, however, from the perspective of the caller, the experience appears to be an interaction with an intelligent machine and the caller is aware that the agent is automated, not human. Accordingly, a typical caller will have a more relaxed expectation of agent responsiveness since a machine, and not an actual person, is on the line.
• FIG. 3A is a block diagram showing asystem30 for providing a message-based communications infrastructure forautomated call center11 operation, also referred to as the Messaging Platform for Agent-Customer Transactions (MPACT)system30, in accordance with the present invention. During regular operation, theMPACT system30 executes multiple threads to process multiple simultaneous calls, which are handled by one or more agents executingagent applications43 on agent consoles39 (shown inFIG. 1). Alternatively, in a further embodiment,multiple MPACT systems30 execute in parallel to provide enhanced performance through loosely- or tightly-coupled parallel processing.
• TheMPACT system30 consists of the following components:database34, telephony interface (TI)35, one or more speech recognition engines (SREs)36, one or more text-to-speech (TTS)engines37,audio cache38, one or more agent consoles39, and optionally,resource manager40. At least one instance of each component is generally required for theMPACT system11 to operate, except that use of theresource manager40 is optional, and is required only on larger systems that incorporate more than one instance of the other components.
• In the described embodiment, amessaging server31,database34,telephony interface35,SREs36,TTS engines37 andaudio cache38 execute on a single computer system while one or more agent consoles39 executing in parallel on separate computer systems. The different components communicate over an Internet Protocol (IP) network, which typically is implemented over high-speed local Ethernet. TheMPACT system30 components run on Intel/AMD-based servers under the Windows 2000 Server Operating System and Redhat Linux. Eachagent console39 runs on Intel/AMD-based workstations under the Windows 2000 Professional Operating System. Each of the components will now be described in further detail.
• The individual computer systems, includingMPACT system30, are general purpose, programmed digital computing devices consisting of a central processing unit (CPU), random access memory (RAM), non-volatile secondary storage, such as a hard drive, network interfaces, and peripheral devices, including user interfacing means, such as a keyboard and display. Program code, including software programs, and data are loaded into the RAM for execution and processing by the CPU and results are generated for display, output, transmittal, or storage.
• Each component is implemented as is a computer program, procedure or module written as source code in a conventional programming language, such as the C++ programming language, and is presented for execution by the CPU as object or byte code, as is known in the art. Alternatively, the components could also be implemented in hardware, either as integrated circuitry or burned into read-only memory components. The various implementations of the source code and object and byte codes can be held on a computer-readable storage medium or embodied on a transmission medium in a carrier wave. TheMPACT system30 operates in accordance with a sequence of process steps, as further described below with reference toFIG. 8.
Messaging Server
• Themessaging server31 consists of ascript engine32 andsession manager47. Thescript engine32 executesscripts49 incorporating agrammar33 that defines a set of executable instructions for specified and interactive question-and-response dialog, and asession manager47 that includes amessage queue48 for staging transient user and agent messages. Script pointers (“Ptrs”)28 that identify theactual scripts49 to be executed by thescript engine32 are maintained locally by eachagent application43. Alternatively,scripts29 could be stored in thedatabase34 and thescript pointers28 would provide database pointers to thescripts29. Themessaging server31 receives call control information from thetelephony interface35 and tracks logged-off, logged-on and signed-on agents. Themessaging server31 uses this information to establish sessions between agents and customers, as further described below with reference toFIG. 4.
• Thedatabase34 contains three primary specialized databases: log database (LDB)44, agent profile database (APDB)45, and agent application database (AADB)46. Thelog database44 provides a running journal as a structured log of each accepted call and stores each user message and agent message in a uniquely-identified record. Thus, agents are freed from the task of having to manually transcribe a telephone conversation following wrap-up. Theagent profile database45 allows themessaging server31 to authenticate, log-on and sign-on agents into registered sessions. Theagent application database46 containsagent applications43 that are executed on agent consoles39. Optionally, thedatabase34 can also containscripts29. Other forms of specialized databases are possible, as would be recognized by one skilled in the art. Alternatively, the information stored in thelog database44,agent profile database45, andagent application database46 could be maintained in structured or unstructured form using a standard file, spreadsheet, or other data assemblage for information storage and retrieval, as is known in the art.
• Definitionally, a signed-on agent is registered on themessaging server31 and is actively processing calls. A logged-on agent is registered on themessaging server31 but is not accepting calls. A logged-off agent is not registered on themessaging server31.
• For each session, themessaging server31 receives customer calls through thetelephony interface35 and sends a stream of transcribed speech utterances as user messages to an agent assigned to handle the session. Note that one or more agents can be assigned to handle any given session and a hierarchy of areas of responsibility, such as speech transcription, customer interaction, controlling scripts, and so forth, can be delegated among several agents to ensure efficient call processing. Similarly, themessaging server31 receives a stream of synthesized speech utterances as agent messages from an assignedagent application43 and sends the agent messages to the customer through thetelephony interface35. The messages typically only contain digitized voice; however, Simultaneous Voice and Data (SVD), for example, Caller ID, can also be provided. The multiplexing and demultiplexing of SVD messages is handled at thetelephony interface35 and anagent console39.
• Thescript engine32 executesindividual scripts49, which incorporate apre-defined grammar33. Thegrammar33 specifies a set of instructions that can be used to create question-and-answer dialogs that are executable by an agent via theagent application43 and thereby enables an agent to process simultaneously multiple calls. Thescripts49 are submitted by agents via anagent console39 using thescript pointers28 during processing of customer calls, as further described below with reference toFIG. 6. Eachscript49 defines a sequence of synthesized speech utterances sent to customers and transcribed speech responses received back as user messages. The speech utterances could be pre-recorded and staged in theaudio cache38. The user messages are used to populate a form (not shown) that is reviewed by the agent during service provisioning.
• FIG. 3B is a block diagram showing asystem160 for providing a message-based communications infrastructure for automated call center operation, in accordance with a further embodiment the present invention. Ascripting server161 executes as a separate system from themessaging server31, which preferably includes only thesession manager47 andmessage queue48. Providing the functionality of thescript engine162 on ascripting server161 enhances overall system throughput and performance by delegating script processing on a system separate from themessaging server31.
• Thescripting server161 consists of adedicated script engine162, which executesscripts163 stored locally to thescripting engine161. Thescripts163 also incorporate thegrammar33. Script pointers (“Ptrs”)164 that identify theactual scripts163 to be executed by thescript engine162 are maintained locally by eachagent application43. Alternatively, scripts29 (shown inFIG. 3A) could be stored in thedatabase34 and thescript pointers164 would provide database pointers to thescripts29.
Telephony Interface
• Referring back toFIG. 3A, customer calls are received through thetelephony interface35, which provides the external connection between theMPACT system30 and the telephone company16 (shown inFIG. 1). The primary purpose of thetelephony interface35 is to accept and process conventional telephone signals, including multiplexing, call routing, and queueing, as is known in the art. In the described embodiment, thetelephony interface35 consists of a third party hardware interface and software drivers, plus MPACT proprietary software that connects the third party package to themessaging server31 and, in large systems, also to theresource manager40. TheMPACT system30 supports standard telephony interface cards, such as analog and T1 Dialogic PCI cards. Optionally, thetelephony interface35 includes anaudio cache41 in which pre-recorded “canned” sound bites are stored for efficient playback. These sound bites provide informational and navigational messages to all callers. Optionally, two or more telephony interfaces35 can be used to provide increased user call capacity. Other forms oftelephony interface35 could be used to accommodate various signal carrier mediums, including conventional land lines; radio, satellite or other forms of signal carriers; light wave or sound wave exchange systems; or equivalents thereof, as would be recognized by ones skilled in the art.
Speech Recognition Engine
• User calls consist of ordinary spoken words, which must be transcribed into written text, for display, processing and storage. The purpose of thespeech recognition engine36 is to generate a stream of transcribed speech utterances that are recorded as computer-processable user messages. In the described embodiment, thespeech recognition engine36 consists of third party software and MPACT proprietary software that connects the third party package to theagent application43 and, in large systems, also to theresource manager40. TheMPACT system30 supports Speechworks and Nuance speech recognition engines. Optionally, two or morespeech recognition engines36 can be used to provide increased user call capacity.
• In a further embodiment, a speech recognition engine executes on aclient system18 interfaced to theMPACT system30 over theinternetwork17, or other data transmission means. TheMPACT system30 receives client messages already transcribed into text by theclient system18 for processing in the same manner as live calls received directly through thetelephony interface35.
Text-to-Speech Engine
• Each caller into theMPACT system30 receives feedback in the form of agent messages, which each consist of a stream of synthesized speech utterances. The synthesized speech utterances include greetings, questions, informational responses, and other spoken phrases that the user hears during the course of the session. The purpose of the text-to-speech engine37 is to generate the stream of synthesized speech utterances formed as agent messages, which can be played back as reproducible audio. The text-to-speech engine37 consists of third party software and MPACT proprietary software that connects the third party package to theagent application43 and, in large systems, also to theresource manager40.MPACT system30 supports Speechworks' Speechify text-to-speech. Optionally, the text-to-speech engine37 can include anaudio cache42 that stores pre-recorded “canned” sound bites, which provide efficient delivery of standardized synthesized speech utterances for use in scripts and repetitive agent actions. As well, two or more text-to-speech engines37 can be used to provide increased user call capacity.
• In a further embodiment, a text-to-speech engine executes on theclient system18 interfaced to theMPACT system30 over theinternetwork17, or other data transmission means. TheMPACT system30 sends agent messages to theclient system18 for synthesis into speech. The synthesized speech is heard by the caller on theclient system18 in the same manner as a live call through telephonic means transacted directly through thetelephony interface35.
Agent Console
• Eachagent console39 provides the primary means for direct customer interaction. The primary purpose of eachagent console39 is to execute one ormore agent applications43, stored in theagent application database46, which display both user and agent messages and provide menus of actions that can be executed responsive to agent commands, including script execution, as further described below with reference toFIG. 6. In the described embodiment, one ormore agent applications43 execute on eachagent console39 and one or more agent consoles39 execute in parallel. Alternatively, multiple instances ofagent applications43 can run on a server machine (not shown) and can be accessed by agents at agent consoles39 operating as remote terminals.
• Eachagent application43 implements a graphical user interface (GUI) for the human agent.FIG. 7 is a screen shot showing, by way of example, a set of callcenter service windows91,92,93 generated by thesystem10 ofFIG. 1. Each callservice center window91,92,93 appears in agraphical user interface90 and enables an agent to indirectly interact with a customer calling through thetelephony interface35. Following sign-on, via theagent application43, an agent can accept new session requests from themessaging server31 and create a visual session container for each session.
• In the described embodiment, up to four sessions can be presented to an agent simultaneously. Preferably, the agent can view the contents of all sessions on a single screen. One session is designated as the active session and accepts agent commands, such as an instruction to listen to a transcribed user message, play a synthesized agent message to the customer, or activate a script through amenu94, as further described below with reference toFIG. 6. The agent can switch active sessions with a single keystroke or pointer click.
• Referring back toFIG. 3A, eachagent application43 receives a stream of transcribed speech utterances from thetelephony interface35 via themessaging server31. In an alternate embodiment (not shown), the stream of transcribed speech utterances bypasses themessaging server31 and is received directly from thetelephony interface35. Themessaging server31 communicates with thespeech recognition engine36 to transcribe the utterances before sending the transcribed utterances to theagent application43. In turn, theagent application43 sends agent messages to thetelephony interface35 via themessaging server31. Themessaging server31 communicates with the text-to-speech engine37 to convert an agent message into a stream of synthesized speech utterances prior to forwarding to thetelephony interface35.
• FIG. 4 is a process flow diagram showingvariable automation levels50 provided using thesystem30 ofFIG. 3A. A typical caller sequence proceeds in three stages:input51, processing52, andoutput53. During theprocessing stage52, thesystem30 provides each agent with a “sliding”control64 that can vary the level of automation used in customer service provisioning. At one end of the slidingcontrol64, the agent must manually type each written response to a user inquiry and, at the other end, thesystem30 directly carries out customer interactions in a fully-automated fashion.
• The slidingcontrol64 accommodates the need to provide linear and non-linear processing to flexibly and dynamically tailor call transaction processing. Non-linear processing occurs when a caller departs from an expected course of action, such as by providing a tangential response to a question in a script requesting specific information. The agent would read the user message and alter the course of script processing to accommodate the tangential response, thereby enhancing the comfort level of the customer towards problem resolution. Linear processing occurs when thesystem30 interacts directly with the caller through pre-defined scripts and is able to successfully complete a series of steps towards problem resolution along a pre-defined course of action. During a course of a call, both linear and non-linear processing can be used strategically to increase user confidence level and to sufficiently process a larger volume of caller traffic than possible with a fully manual and fully non-linear call processing approach.
• During theinput stage51,speech54 is received into the automated call center11 (shown inFIG. 1), either by ahuman operator55 or amachine56, such as theMPACT system30. Thespeech54 is converted into a stream of transcribed speech utterances or “normal” text annotations57. The text annotations57 are machine-processable as inputs to theprocessing stage52. A message manager (not shown) associates incoming the audible speech utterances with thetext annotations57 and the associated incoming audible speech utterances are provided with thetext annotations57 to the agent for optional playback.
• Processing generally involves the execution of some form of action, such as a script execution, as further described below with reference toFIG. 5. During theprocessing stage52, either ahuman operator58, that is, an agent, or amachine59 interprets eachannotation57 and generates a response orquery60. The response orquery60 is received in theoutput stage53, either by ahuman operator61, that is, an agent, or amachine62. Finally, the response orquery60 is formed into synthesizedspeech utterances63 that are played back to the caller.
• In the described embodiment, the three stages of call processing,input51, processing52, andoutput53, are performed bymachines56,59,62, although adjunctive processing can be performed byhuman operators55,58,61, as necessary to supplement theMPACT system30. Thus, a slidingcontrol64 of human operator and machine interaction can be provided to automate call center operations. Using the slidingcontrol64, the agent can change the behavior of the script engine32 (shown inFIG. 3A) by beginning execution of ascript29, adjusting the point of execution within ascript29, or by causing a portion of thescript29 to be repeated. The agent can thereby alter the ordinarily sequential control flow of script processing by intervening as necessary, based on the current context of the call, or can allow script processing to proceed in a linear fashion.
• In addition to indirect communication via anagent console39, each agent can also accept live calls from customers directly through thetelephony interface35 in a real time, voice-to-voice mode. Live call processing may be required to handle crises or other service needs that are not amenable to automation, or to provide those services to customers, which are not generally acceptable when provided via automation.
• In a further embodiment, the agent communicates with callers executing onclient systems18 through text message exchange transacted over theinternetwork17, or other data transmission means. Unlike conventional chat sessions, caller processing is transacted in the same manner in which telephonic calls received through thetelephony interface35 are transacted. Consequently, the agent can apply the slidingcontrol64 over automation level to ensure a continuum of non-linear to linear call processing independent of the actual call transmission means. Various arrangements and combinations of call transmission means can therefore be accommodated, as would be recognized by one skilled in the art.
• Referring back toFIG. 3A, as a first step, an agent, via anagent console39, authenticates to themessaging server31 prior to logging on. Following log-on, the agent indicates availability to handling calls by signing on to themessaging server31. Thereafter, calls that have been accepted into the automated call center11 (shown inFIG. 1) are assigned to a session, and the session is subsequently assigned to a signed-on agent.
• After the initial communication between the agent and the customer, anagent application43 ordinarily loads a script describing the session flow from theapplication database45, either according to agent instructions or based on information provided by thetelephony interface35, such as Caller ID. Theagent application43 communicates further via themessaging server31 with thespeech recognition engine36 and text-to-speech engine37 to transcribe text-to-speech and convert speech-to-text, as necessary. Communication between theagent application49 and thespeech recognition engine36 and text-to-speech engine37 continues until the call terminates.
• FIG. 5 is a block diagram showing aschema70 for storing records in thelog database44 of theMPACT system30 ofFIG. 3A. Each session is identified by a session identifier (ID)71, which uniquely identifies each session, such as a sequentially-increasing number. In addition, each record contains a message identifier (ID)72, time stamped73, sender orreceiver flag74, andcontent75, typically constituting audio data, text data or notes transcribed by the agent. Other types of identifiers, formation and content can be stored in a record, as would be recognized by one skilled in the art.
• Referring back toFIG. 3A, eachagent application43 also contains a supervisor mode that can be used to manually monitor system behavior and performance,control agent application43 andmessaging server31 operation, monitor and guide human agents actions, and perform similar administrative tasks. A separate administrator application (not shown) enables system setup and configuration.
• Finally, eachagent application43 can executescripts49 to perform a pre-defined sequence of instructions, generally consisting of question-and-response dialogues through which a customer is queried regarding a concern or to troubleshoot a problem.FIG. 6 is a process flow diagram showing, by way of example, thecontrol flow80 followed in processing ascript49 using thesystem30 ofFIG. 3A. During the initial stage of processing, a customer82 indirectly interacts with anagent81 through the execution of aninitial script83. The purpose of theinitial script83 is to populate astandard form84 with general information regarding the nature of the call. Upon reviewing theform84, theagent81 executes, via theagent application43, one or moreadditional scripts87 to provide problem resolution or troubleshooting and to receive further information viaadditional forms88. Finally, the agent, via theagent application43, generates aresolution form86 that is used as parameters to aclosing script85, which is executed for playback to the customer82 to complete the call.
• In the described embodiment, eachform84,86,88 is structured as a data containment object that stores data relevant to theagent application43. Preferably, each data containment object is maintained as a binary large object (BLOB) interpretable by eachagent application43 based on business requirements. Significantly, the use of scripts, forms and agent interaction enables a non-linear execution path through problem resolution and troubleshooting. As necessary, an agent, through theagent application43, can manually enter data into a form and progressively modify the sequence of problem resolution and troubleshooting. The amount of manual agent intervention follows from the slidingcontrol64 implemented in theMPACT system30, as described above with reference toFIG. 4.
• Referring back toFIG. 3A, thescript engine32 executes eachscript49, which incorporate thegrammar33. By way of example, a sample grammar for use in the described embodiment is provided in Appendix A. Other forms of grammars and scripting languages could be used, as would be recognized by one skilled in the art.
Resource Manager
• Theresource manager40 provides scalability, load balancing and redundancy in large systems comprising multiplespeech recognition engines36, text-to-speech engines37, and telephony interfaces35. In the described embodiment, themessaging server31 has a built-in simple resource manager40 (not shown) to managemultiple agent applications43 operating in smaller call capacity systems.
• Using an administration application that controls theresource manager40, an administrator can set and configure the system while operational. Theresource manager40 enables the administrator to add or remove servers and to reroute connections between different components, for instance, betweentelephony interface35,messaging server31 andagent application43.
Audio Cache
• Theaudio cache38 provides a centralized repository in which pre-recorded “canned” sound bites are stored for efficient playback. These sound bites provide both informational and navigational messages to all callers and standardized synthesized speech utterances for use in scripts and repetitive agent actions. The sound bites in theaudio cache38 are retrievable by thetelephony interface35, text-to-speech engine37 andagent application43.
System Configuration and Capacity
• In a typical system, different components run on separate machines. A typical medium-sized system consists of one server running atelephony interface35 andmessaging server31, a separate server for thespeech recognition engine36, another server for the text-to-speech engine37, and a fourth server for the log, agent profile, andagent application databases44,45,46, respectively. Alternatively, a minimal system runs all the components on a single server, along with anagent application43 on anintegrated agent console39.
• In the described embodiment, each medium-sized system configuration is “24×6,” meaning that the system can handle 24 simultaneous calls and can interface with six human agents. A minimal system configuration is “4×1,” that is, four simultaneous calls with one human agent, while a large system configuration is “96×24,” that is, 96 simultaneous calls and 24 human agents. Through theresource manager40, an aggregation of the above-described configurations enables much larger call capacity systems.
• By mixing proprietary technologies and existing systems, theMPACT system30 reduces the average call time by 30%, increases agent efficiency up to four-fold, and significantly improves customer satisfaction.
Method for Providing a Message-Based Communications Infrastructure
• FIG. 8 is a flow diagram showing a method for providing a message-basedcommunications infrastructure100 for automated call center operation, in accordance with the present invention. The method is executed by theMPACT system30 and individual operations are executed by the various components, specifically described below. During regular operation, theMPACT system30 processes multiple simultaneous calls, which are handled by one or more agents executingagent applications43 on an agent console39 (shown inFIG. 1).
• Generally, themethod100 proceeds by iteratively processing each call in a continuous processing cycle. During each cycle, a call is received (block101) and assigned to a session (block102) by the session manager47 (shown inFIG. 3A), as further described below with reference toFIG. 9. Next, the session is processed (block103), as further described below with reference toFIG. 10. Following session processing, the call ends (block104) and further call are processed (block105) until theMPACT system30 is shut down or no further calls are received. The method then terminates. In the described embodiment, the MPACT is a multi-threaded system, employing multiple threads, which each independently execute themethod100.
• FIG. 9 is a flow diagram showing a routine for signing asession110 for use in themethod100 ofFIG. 8. The purpose of the routine is to instantiate a new session object for processing by an agent application43 (shown inFIG. 3A) and to subsequently assign the instantiated session to an agent operating on anagent console39.
• Initially, if any agent is available (block111), one of the agents is picked as the assigned agent (block112) and the new session is created (block113). Subsequently, the selected agent is assigned to the newly-created session (block114). The routine then returns. If no agents are available (block111), the customer is presented with the option of interfacing to a non-agent (block115), that is, an automated voice response system, which provides the information specifically requested by the customer (block116), after which the routine returns. Otherwise, if the customer prefers an agent (block115), the customer enters into a waiting queue (block117) until an agent becomes available.
• Note that both the customers and agents can be prioritized using predefined selection criteria. For instance, customers who have enrolled in premium support service can received a higher priority in the waiting queue than other customers. As well, specialized problem-resolution agents can be prioritized for servicing particular customer needs for consideration during selection of agent assignment.
• FIG. 10 is a flow diagram showing a routine125 for processing asession120 for use in themethod100 ofFIG. 8. The purpose of the routine is to iteratively store and forward messages using the message queue48 (shown inFIG. 3A). Other forms of queueing and message handling and prioritization are possible, as would be recognized by one skilled in the art.
• During each iteration, the session manger47 (shown inFIG. 3A) waits for a message, either a user message or agent message (block121). Upon receipt, the message is placed in the message queue48 (block122) and the recipient of the message is notified (block123). If the message is a user message being sent to an agent (block124), the message is processed by the agent assigned to the session to which the user message corresponds (block125), as further described below with reference toFIG. 11. Iterative processing continues with each subsequent message (block126), after which the routine returns.
• FIG. 11 is a flow diagram showing a routine for processing a session by anagent130 for use in the routine125 ofFIG. 10. The purpose of the routine is to facilitate the interaction between an agent and customer though an agent application executing on an agent console39 (shown inFIG. 3A).
• First, the notification message is displayed (block131) on the graphical user interface90 (shown inFIG. 7) of theagent application43. As necessary, the agent sends agent messages to the customer from service provider or script engine32 (shown inFIG. 3A) to provide appropriate handling of the user message (block132). If the sent message consists of a request to execute a script49 (block132), the message is further processed by the script engine32 (block134), as further described below with reference toFIG. 12. The routine then returns.
• FIG. 12 is a flow diagram showing a routine for processing a session by ascript engine140 for use in the routine130 ofFIG. 11. The purpose of this routine is to iteratively process each script execution request using thescript engine32 on behalf of requesting agent applications43 (shown inFIG. 3A).
• Each message thread is iteratively processed (blocks141-146) as follows. During each iteration (block141), the notes field of each message is retrieved (block142) and, if a script execution request is found (block143), ascript49 is executed (block144), as further described below with reference toFIG. 13. Otherwise, if no script request is present (block143), no operation occurs (block145). Processing continues with each additional message thread (block146), after which the routine returns.
• Although described above with reference to the linear processing of a script in sequential order, agent intervention in script processing is fully supported. The agent continues to monitor the progress of the script execution by observing user responses and can intervene as necessary to accommodate a non-scripted response. For example, the user may provide a tangential response to a question in the script requesting specific information. The agent would read the user message and alter the course of script processing to accommodate the tangential response to the sliding control64 (shown inFIG. 4).
• FIG. 13 is a flow diagram showing a routine for executing ascript150 for use in the routine140 ofFIG. 12. The purpose of this routine is to perform standard retrieve-and-interpret script instruction execution, as is known in the art.
• First, each instruction is retrieved (block151) and executed (block152). In the described embodiment, instruction execution follows from an interpretable stored grammar33 (shown inFIG. 3A) and as provided, by way of example, in Appendix A. If the executing instruction is a “Quit” instruction (block153), the script execution terminates and the routine returns. Otherwise, script processing continues with the next instruction (block154).
System for Providing a Message-Based Communications Infrastructure for Automated Call Center Post-Call Processing
• FIGS. 14A and 14B are process flow diagrams170,180 showing, by way of example, a typicalpost-call sequence170, as transacted in the automated call centeroperational environment10 ofFIG. 1. Referring first toFIG. 14A, a similar sequence of initial processes are followed for incoming calls, as previously described above with reference toFIG. 2. Thus, upon calling into theautomated call center11, each user receives an initial greeting and engages in acustomer support scenario22 with an agent. As required,service provisioning23 is provided to the user and the call ends in a wrap-up24, which provides closure to the call and a departing salutation.
• Following call termination, the system can performpost-call processing171, which identifies individual speech utterances in each call that are stored into thedatabase34. The identified speech utterances can be presented to an agent for manipulation, such as ranking or reordering. In addition, the processing can include performing speech recognition on the speech utterances, identifying speaker characteristics, and marking certain speech utterances for later use. In a further embodiment, the system can perform in-progress call processing172, which operates on a stream of speech utterances copied from on-going calls. The same types of processing can be performed as on completed calls, but further allow real time call analysis.
• Referring next toFIG. 14B, completed outgoing calls are processed in a slightly different manner than incoming calls where the outgoing call originated with the system. Certain aspects of an outgoing call, such as the characteristics of the originating caller and the general flow of the conversation, are already known. This knowledge can be utilized to streamline call processing and subsequent repeat calls to the some type of callee. For example, amanual call181, originating at the call center, could generate a completed call that can then undergopost-call processing182 in a manner similar to thepost-call processing171 for incoming calls. However, the pre-calling information can be used inmacro generation183, which automates future outgoing calls to callers sharing similar characteristics as themanual call181. For instance, an answering machine or user call menu may require little to no manual agent interaction until a certain point in the call, and would therefore be amenable to an automated calling approach provided through system-generatedmacro execution184. Other types of processing of incoming and outgoing calls are possible, as would be appreciated by one skilled in the art.
• FIG. 15 is a block diagram showing asystem190 for providing a message-based communications infrastructure forautomated call center11 post-call processing, also referred to as theMPACT system190, in accordance with the present invention. TheMPACT system190 includes a similar set of components, as previously described above with reference toFIG. 3A, but themessaging server31 is supplemented with apost-call processor191 and eachagent application43 is supplemented with a command processor193 (“cmd proc”).
• Thepost-call processor191 can be triggered to process completed and, in a further embodiment, in-progress calls. Each call is initially scanned and avoice parser192 identifies individual verbal speech utterances within a call. The identified speech utterances are stored asuser messages194 in thedatabase34, along with any annotations (not shown) created to accompany the written record of the call.
• Thecommand processor193 executes as part of theagent application43 to present theuser messages194 to an agent on theagent console39 and enables the agent to execute commands to process theuser messages194. The processing can include performing speech recognition on the verbal utterances, identifying characteristics of one or more of the speakers participating in a call by, for instance, role, such as agent or third party, gender, and language. Other speaker characteristics are possible. Thecommand processor193 can also allow the agent to mark certain speech utterances for future use. In a further embodiment, speech recognition can be performed either before or after identifying speaker characteristics or marking speech utterances, or as necessary, such as when identifying a spoken language or accent. The results of the processing are stored into thedatabase34. The additional components of theMPACT system190, that is, thepost-call processor191 andcommand processor193, operate in accordance with a sequence of process steps, as further described below beginning with reference toFIG. 16.
Method for Providing a Message-Based Communications Infrastructure for Automated Call Center Post-Call Processing
• FIG. 16 is a flow diagram showing a method for providing a message-basedcommunications infrastructure200 for automated call center post-call processing, in accordance with the present invention. The method is executed by theMPACT system190 and individual operations are executed by the various components, specifically described below.
• Generally, themethod200 proceeds by iteratively processing each call in a continuous processing cycle. During each cycle, a call is first retrieved (block201). A completed call could be transiently stored in a cache or database while an in-progress call could be staged in an interim buffer. Once retrieved, the call is parsed (block202) and individual verbal speech utterances are identified (block203). The call is then stored into the database34 (block204) and processed (block205), as further described below with reference toFIG. 17. The method then terminates. In the described embodiment, the MPACT is a multi-threaded system, employing multiple threads, which each independently execute themethod100.
• FIG. 17 is a flow diagram showing a routine210 for processing a call for use in themethod200 ofFIG. 16. The purpose of the routine is to perform optional types of processing on the verbal speech utterances extracted from each call.
• Speech recognition is performed on one or more of the verbal speech utterances (block211). Characteristics of one or more of the speakers can be identified (block212), such as by analyzing speech pattern, inflection, tonality, accent, pitch, and similar attributes, which provide clues as to the role, gender, and language of the speaker. Other speaker characteristics are possible. Individual speech utterances can also be marked for future use (block213). In a further embodiment, speech recognition can be performed either before or after identifying speaker characteristics or marking speech utterances, or as necessary, such as when identifying a spoken language or accent. In a further embodiment, macros can be generated for automating outgoing calls (block214). Finally, the characteristics and speech utterance markings are stored in the database34 (block215). The routine then returns.
• While the invention has been particularly shown and described as referenced to the embodiments thereof, those skilled in the art will understand that the foregoing and other changes in form and detail may be made therein without departing from the spirit and scope of the invention.
APPENDIX
• To specify an alert message string, use:
• ALERT string
• To specify an unconditional branch to label, use:
• GOTO label
• To specify a conditional cond branch to label, use:
• IF cond THEN GOTO label
• To specify a label string for a branch, use:
• LABEL string
• To terminate execution, use:
• QUIT
• To specify a synthesized speech utterance string, use:
• SAY string
• To specify a script name string, use:
• SCRIPTNAME string
• To specify a variable name varname, use:
• SET varname
• To specify a subscript scriptname, use:
• SUBSCRIPT scriptname
• To specify a wait condition string, use:
• WAITFOR string <YES|NO|DATE|PHONENUM|CUSTID>

Claims (20)

What is claimed is:

1. A computer-implemented system for facilitating call sessions via messages, comprising:

a call module to monitor two or more call sessions;

a call processor to process each call session comprising:

a transcriber to receive speech utterances from one of the participants, to transcribe the speech utterances into text, and to provide the transcribed speech utterances to another participant in the call session; and

a speech module to receive text messages from the other participant in response to the transcribed speech utterances and to provide to the participant synthesized speech based on the text messages from the other participant; and

a log module to generate during each call session an ongoing log of messages comprising the transcribed speech utterances and the text messages for that call session and to add to the log annotations from one or more of the participants to that call session.

2. A system according toclaim 1, further comprising:

a marker to mark one or more of the transcribed speech utterances in the log for later processing.

3. A system according toclaim 1, further comprising:

a database to store with each of the transcribed speech utterances and text messages a message identifier, time, sender, and receiver.

4. A system according toclaim 1, further comprising:

a display to display the ongoing log to at least one of the participants during one such call session.

5. A system according toclaim 1, further comprising:

a notification module to notify at least one of the participants upon receipt of one such message from the remaining participant for one such call session.

6. A system according toclaim 1, further comprising:

a database to store a set of speech utterances; and

identifying within one such call session, those speech utterances from the participant that match the stored speech utterances.

7. A system according toclaim 6, further comprising:

a processing module to perform one of ranking and reordering of the identified speech utterances from the participant.

8. A system according toclaim 1, further comprising:

a characteristic module to identify characteristics of the participant from the speech utterances.

9. A system according toclaim 1, further comprising:

a switch module to allow the other participant to switch to a different call session with a different participant.

10. A system according toclaim 1, wherein the synthesized speech comprises one or more of pre-recorded sound bits and text-to-speech generated messages based on the text messages from the other participant.

11. A computer-implemented method for facilitating call sessions via messages, comprising:

monitoring two or more call sessions;

processing each call session comprising:

receiving speech utterances from one of the participants and transcribing the speech utterances into text;

providing the transcribed speech utterances to another participant in the call session; and

receiving text messages from the other participant in response to the transcribed speech utterances and providing to the participant synthesized speech based on the text messages from the other participant;

generating during each call session an ongoing log of messages comprising the transcribed speech utterances and the text messages for that call session; and

adding to the log annotations from one or more of the participants to that call session.

12. A method according toclaim 11, further comprising:

marking one or more of the transcribed speech utterances in the log for later processing.

13. A method according toclaim 11, further comprising:

storing with each of the transcribed speech utterances and text messages a message identifier, time, sender, and receiver.

14. A method according toclaim 11, further comprising:

displaying the ongoing log to at least one of the participants during the call session.

15. A method according toclaim 11, further comprising:

notifying at least one of the participants upon receipt of one such message from the remaining participant for one such call session.

16. A method according toclaim 11, further comprising:

storing a set of speech utterances; and

identifying within one such call session, those speech utterances from the participant that match the stored speech utterances.

17. A method according toclaim 16, further comprising:

performing one of the ranking and reordering the identified speech utterances from the participant.

18. A method according toclaim 1, further comprising:

identifying characteristics of the participant from the speech utterances.

19. A method according toclaim 11, further comprising:

allowing the other participant to switch to a different call session with a different participant.

20. A method according toclaim 11, wherein the synthesized speech comprises one or more of pre-recorded sound bits and text-to-speech generated messages based on the text messages from the other participant.

Images