USRE48846E1 - Estimating agent performance in a call routing center system - Google Patents

Abstract

Systems and methods are disclosed for estimating and assigning agent performance characteristics in a call routing center. Performance characteristics (e.g., sales rate, customer satisfaction, duration of call, etc.) may be assigned to an agent when the agent has made few calls relative to other agents or otherwise has a large error in their measure of one or more performance characteristics used for matching callers to agents (e.g., via a performance based or pattern matching routing method). A method includes identifying agents of a plurality of agents having a number of calls fewer than a predetermined number of calls (or an error in the performance characteristic exceeding a threshold), assigning a performance characteristic to the identified agents (that is different than the agent's actual performance characteristic), and routing a caller to one of the plurality of agents based on the performance characteristics of the plurality of agents.

Description

Note: More than one reissue patent application has been filed for the reissue of U.S. Pat. No. 8,724,797. The reissue patent applications are the present U.S. Reissue patent application Ser. No. 15/153,553, filed May 12, 2016, and U.S. Reissue patent application Ser. No. 15/210,763, filed Jul. 14, 2016, and U.S. Reissue patent application Ser. No. 15/210,779, filed Jul. 14, 2016, each of which is a continuation reissue application of the present U.S. Reissue patent application Ser. No. 15/153,553, filed May 12, 2016.

BACKGROUND

1. Field

The present invention relates generally to the field of routing phone calls and other telecommunications in a call routing center system, and in particular, to methods and system for matching callers to agents based on performance and pattern matching algorithms.

2. Related Art

The typical contact center consists of a number of human agents, with each assigned to a telecommunication device, such as a phone or a computer for conducting email or Internet chat sessions, that is connected to a central switch. Using these devices, the agents are generally used to provide sales, customer service, or technical support to the customers or prospective customers of a contact center or a contact center's clients.

Typically, a contact center or client will advertise to its customers, prospective customers, or other third parties a number of different contact numbers or addresses for a particular service, such as for billing questions or for technical support. The customers, prospective customers, or third parties seeking a particular service will then use this contact information, and the incoming caller will be routed at one or more routing points to a human agent at a contact center who can provide the appropriate service. Contact centers that respond to such incoming contacts are typically referred to as “inbound contact centers.”

Similarly, a contact center can make outgoing contacts to current or prospective customers or third parties. Such contacts may be made to encourage sales of a product, provide technical support or billing information, survey consumer preferences, or to assist in collecting debts. Contact centers that make such outgoing contacts are referred to as “outbound contact centers.”

In both inbound contact centers and outbound contact centers, the individuals (such as customers, prospective customers, survey participants, or other third parties) that interact with contact center agents using a telecommunication device are referred to in this application as a “caller.” The individuals acquired by the contact center to interact with callers are referred to in this application as an “agent.”

Conventionally, a contact center operation includes a switch system that connects callers to agents. In an inbound contact center, these switches route incoming callers to a particular agent in a contact center, or, if multiple contact centers are deployed, to a particular contact center for further routing. In an outbound contact center employing telephone devices, dialers are typically employed in addition to a switch system. The dialer is used to automatically dial a phone number from a list of phone numbers, and to determine whether a live caller has been reached from the phone number called (as opposed to obtaining no answer, a busy signal, an error message, or an answering machine). When the dialer obtains a live caller, the switch system routes the caller to a particular agent in the contact center.

Routing technologies have accordingly been developed to optimize the caller experience. For example, U.S. Pat. No. 7,236,584 describes a telephone system for equalizing caller waiting times across multiple telephone switches, regardless of the general variations in performance that may exist among those switches. Contact routing in an inbound contact center, however, is a process that is generally structured to connect callers to agents that have been idle for the longest period of time. In the case of an inbound caller where only one agent may be available, that agent is generally selected for the caller without further analysis. In another example, if there are eight agents at a contact center, and seven are occupied with contacts, the switch will generally route the inbound caller to the one agent that is available. If all eight agents are occupied with contacts, the switch will typically put the contact on hold and then route it to the next agent that becomes available. More generally, the contact center will set up a queue of incoming callers and preferentially route the longest-waiting callers to the agents that become available over time. Such a pattern of routing contacts to either the first available agent or the longest-waiting agent is referred to as “round-robin” contact routing. In round robin contact routing, eventual matches and connections between a caller and an agent are essentially random.

Some attempts have been made to improve upon these standard yet essentially random processes for connecting a caller to an agent. For example, U.S. Pat. No. 7,209,549 describes a telephone routing system wherein an incoming caller's language preference is collected and used to route their telephone call to a particular contact center or agent that can provide service in that language. In this manner, language preference is the primary driver of matching and connecting a caller to an agent, although once such a preference has been made, callers are almost always routed in “round-robin” fashion.

BRIEF SUMMARY

According to one aspect of the present invention, a process is described for assigning agents in a call center with one or more performance characteristics for use in a routing process based on agent performance. Broadly speaking, and in one example, agent performance characteristics (e.g., sales rates, customer satisfaction scores, duration of calls, etc.) may be assigned to an agent when the agent has made few (or no) calls relative to other agents or otherwise has a large error in their measure of one or more performance characteristics used for matching callers to agents (e.g., via a performance based or pattern matching routing method). As an agent makes more calls, or the error decreases, the assigned agent performance characteristic may be adjusted or returned to an actual performance characteristic for the agent.

In one example, a method includes identifying agents of a plurality of agents (e.g., all agents of a call routing center or a subset thereof) having a number of calls fewer than a predetermined number of calls. The number of calls may be based on various factors relating to fractional errors, relative to an average number of calls, selected and adjusted by a call center manager, and so on. The method further includes assigning a performance characteristic to the identified agents that is different than the agent's actual performance characteristic. Callers are then routed or matched to agents of the plurality of agents based, at least in part, on the performance characteristics of the plurality of agents (including the assigned performance characteristics of those identified having a number of calls fewer than the predetermined number of calls and the actual performance characteristic of the other agents).

The assigned performance characteristic may include, or be based on, an average or historical performance characteristic of the plurality of agents. In some examples, the assigned performance characteristic may be based on an agent's demographic data, e.g., using actual agent performance of agents having similar demographic data, to estimate performance. In other examples, the assigned agent performance characteristic may include an adjustment to an actual performance characteristic of the one or more identified agents. The adjustment may include an interpolation between an actual performance characteristic of the identified agent and an average performance characteristic of the plurality of agents.

In another example, a method includes determining performance characteristics for a plurality of agents, determining an average performance characteristic for the plurality of agents, and determining an error (e.g., a fractional or percentage error) in the performance characteristic for each of the plurality of agents. Agents having an error greater than a predetermined threshold may be assigned a performance characteristic different than their actual performance characteristic. The method further including routing a caller to one of the plurality of agents based on the performance characteristics of the plurality of agents and the assigned performance characteristic of the identified agent(s).

In one example, exemplary routing or matching methods using the actual and assigned performance characteristics may include performance based matching (e.g., ranking a set of agents based on performance and preferentially matching callers to the agents based on a performance ranking or score), pattern matching algorithms (e.g., comparing agent data associated with a set of callers to agent data associated a set of agents and determining a suitability score of different caller-agent pairs), affinity data matching, and other models for matching callers to agents. The methods may therefore operate to output scores or rankings of the callers, agents, and/or caller-agent pairs for a desired optimization (e.g., for optimizing cost, revenue, customer satisfaction, and so on).

Additionally, an interface may be presented to a user allowing for adjustment of various features of the routing system, e.g., a slider or selector for adjusting the threshold values, performance characteristic(s), and so on, in real-time or a predetermined time. The interface may allow a user to turn certain methods on and off, change desired optimizations, and may display an estimated effect of a change in a threshold value (e.g., number of calls or error).

Many of the techniques described here may be implemented in hardware, firmware, software, or combinations thereof. In one example, the techniques are implemented in computer programs executing on programmable computers that each includes a processor, a storage medium readable by the processor (including volatile and nonvolatile memory and/or storage elements), and suitable input and output devices. Program code is applied to data entered using an input device to perform the functions described and to generate output information. The output information is applied to one or more output devices. Moreover, each program is preferably implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram reflecting the general setup of a contact center operation.

FIG. 2 illustrates an exemplary routing system having a routing engine for routing callers based on performance and/or pattern matching algorithms.

FIG. 3 illustrates an exemplary routing system having a mapping engine for routing callers based on performance and/or pattern matching algorithms.

FIG. 4 illustrates an exemplary method or computer model for matching callers to agents based on performance.

FIG. 5 illustrates an exemplary method or computer model for matching callers to agents based on caller data and agent data.

FIG. 6 illustrates an exemplary method or computer model for estimating a performance characteristic of an agent.

FIG. 7 illustrates another exemplary method or computer model for estimating a performance characteristic of an agent.

FIGS. 8A-8E illustrate graphs for an exemplary Monte Carlo simulation for selecting a desired threshold for assigning performance characteristics.

FIG. 9 illustrates a typical computing system that may be employed to implement some or all processing functionality in certain embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The following description is presented to enable a person of ordinary skill in the art to make and use the invention, and is provided in the context of particular applications and their requirements. Various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Moreover, in the following description, numerous details are set forth for the purpose of explanation. However, one of ordinary skill in the art will realize that the invention might be practiced without the use of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order not to obscure the description of the invention with unnecessary detail. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

While the invention is described in terms of particular examples and illustrative figures, those of ordinary skill in the art will recognize that the invention is not limited to the examples or figures described. Those skilled in the art will recognize that the operations of the various embodiments may be implemented using hardware, software, firmware, or combinations thereof, as appropriate. For example, some processes can be carried out using processors or other digital circuitry under the control of software, firmware, or hardwired logic. (The term “logic” herein refers to fixed hardware, programmable logic and/or an appropriate combination thereof, as would be recognized by one skilled in the art to carry out the recited functions.) Software and firmware can be stored on computer-readable storage media. Some other processes can be implemented using analog circuitry, as is well known to one of ordinary skill in the art. Additionally, memory or other storage, as well as communication components, may be employed in embodiments of the invention.

According to one aspect of the present invention, systems, and methods are provided for routing callers to agents within a call center. In one example, a method includes performance based matching (e.g., ranking a set of agents based on performance and preferentially matching callers to the agents based on a performance ranking or score). In a basic example, a performance algorithm includes one or more performance characteristics associated with each agent as input, e.g., an agent's sales or conversion rate, satisfaction rating, cost (e.g., duration of call), and so on. The agent's performance characteristic can also be used in one or more pattern matching algorithms (e.g., comparing agent data, including performance characteristics, associated with a set of agents to caller data associated a set of callers and determine a suitability score of different caller-agent pairs), affinity data matching, and other models for matching callers to agents. The methods may therefore operate to output scores or rankings of the callers, agents, and/or caller-agent pairs for a desired optimization (e.g., for optimizing cost, revenue, customer satisfaction, and so on).

Accurately measuring an agent's performance characteristic(s), however, can be problematic. For instance, in many call routing centers a number of agents may have zero performance, e.g., no conversions or sales, which may occur when an agent first starts and has made few or no calls. This is particularly an issue where the campaign of a call routing center has an intrinsically low sales or Conversion Rate (CR). For instance, if the overall CR of a campaign is low, e.g., 4%, an inexperienced new agent may have to make a large number of calls (e.g., hundreds) before converting a sale (or obtaining some other performance characteristic).

In one example, an exemplary method and system includes identifying agents having a small number of calls, e.g., below a predetermined threshold, and likely a large error in their estimate of an agent performance characteristic, or Agent Performance (AP). In one example, the identified agents are assigned an estimated AP value near or equal to the mean AP for agents across the whole campaign. For example, in the extreme case where an agent has few or no calls, such an agent may be assigned a CR at or near the mean CR of the whole campaign. In other examples, the CR can be assigned at some percentage, e.g., 80% or 90%, of the mean CR of the whole campaign or some other set of agents. As the particular agent makes more calls, the agent's actual CR becomes more reliable (i.e., as they make more calls the error in their estimated AP is reduced), and at a predetermined threshold of calls or error, the system assigns their actual CR value to them. Agents with an intermediate number of calls, e.g., more than zero but less than the predetermined threshold (e.g., sufficient to give a small enough error in their AP) can be assigned an AP between their measured AP and the overall campaign conversion rate. In one example, the system assigns such agents based on a linear interpolation between the agent's measured AP and the overall campaign CR.

Initially, exemplary call routing systems and methods utilizing performance (and/or pattern matching algorithms, either of which may be used within generated computer models for predicting the chances of desired outcomes) are described for routing callers to available agents. This description is followed by exemplary systems and methods for estimating agent performance for use as input into a performance or pattern matching algorithm.

I. Exemplary Systems and Routing Engine

FIG. 1 is a diagram reflecting the general setup of acontact center operation100. Thenetwork cloud101 reflects a specific or regional telecommunications network designed to receive incoming callers or to support contacts made to outgoing callers. Thenetwork cloud101 can comprise a single contact address, such as a telephone number or email address, or multiple contract addresses. Thecentral router102 reflects contact routing hardware and software designed to help route contacts amongcall centers103. Thecentral router102 may not be needed where there is only a single contact center deployed. Where multiple contact centers are deployed, more routers may be needed to route contacts to another router for aspecific contact center103. At thecontact center level103, acontact center router104 will route a contact to anagent105 with an individual telephone orother telecommunications equipment105.

FIG. 2 illustrates an exemplary contact center routing system200 (which may be included withcontact center router104 ofFIG. 1). Broadly speaking,routing system200 is operable to match callers and agents based, at least in part, on agent performance or pattern matching algorithms using caller data and/or agent data.Routing system200 may include acommunication server202 and a routing engine204 (referred to at times as “SatMap” or “Satisfaction Mapping”) for receiving and matching callers to agents (referred to at times as “mapping” callers to agents).

Routing engine204 may operate in various manners to match callers to agents based on performance data of agents, pattern matching algorithms, and computer models, which may adapt over time based on the performance or outcomes of previous caller-agent matches. In one example, therouting engine204 includes a neural network based adaptive pattern matching engine. Various other exemplary pattern matching and computer model systems and methods which may be included with content routing system and/orrouting engine204 are described, for example, in U.S. Ser. No. 12/021,251, filed Jan. 28, 2008, U.S. Ser. No. 12/202,091, filed Aug. 29, 2008, and U.S. Ser. No. 12/266,461, filed Nov. 6, 2008, all of which are hereby incorporated by reference in their entirety. Of course, it will be recognized that other performance based or pattern matching algorithms and methods may be used alone or in combination with those described here.

Routing system200 may further include other components such as collector206 for collecting caller data of incoming callers, data regarding caller-agent pairs, outcomes of caller-agent pairs, agent data of agents, and the like. Further,routing system200 may include areporting engine208 for generating reports of performance and operation ofrouting system200. Various other servers, components, and functionality are possible for inclusion withrouting system200. Further, although shown as a single hardware device, it will be appreciated that various components may be located remotely from each other (e.g.,communication server202 androuting engine204 need not be included with a common hardware/server system or included at a common location). Additionally, various other components and functionality may be included withrouting system200, but have been omitted here for clarity.

FIG. 3 illustrates detail ofexemplary routing engine204.Routing engine204 includes amain mapping engine304, which receives caller data and agent data fromdatabases310 and312. In some examples,routing engine204 may route callers based solely or in part on performance data associated with agents. In other examples,routing engine204 may make routing decisions based solely or in part on comparing various caller data and agent data, which may include, e.g., performance based data, demographic data, psychographic data, and other business-relevant data. Additionally, affinity databases (not shown) may be used and such information received by routingengine204 for making routing decisions.

In one example,routing engine204 includes or is in communication with one or moreneural network engines306.Neural network engines306 may receive caller and agent data directly or viarouting engine204 and operate to match and route callers based on pattern matching algorithms and computer models generated to increase the changes of desired outcomes. Further, as indicated inFIG. 3, call history data (including, e.g., caller-agent pair outcomes with respect to cost, revenue, customer satisfaction, etc.) may be used to retrain or modify theneural network engine306.

Routing engine204 further includes or is in communication withhold queue308, which may store or access hold or idle times of callers and agents, and operates to map callers to agents based on queue order of the callers (and/or agents).Mapping engine304 may operate, for example, to map callers based on a pattern matching algorithm, e.g., as included withneural network engine306, or based on queue order, e.g., as retrieved fromhold queue308.

II. Exemplary Performance Based and Pattern Matching Processes

FIG. 4 illustrates a flowchart of an exemplary method or model for matching callers to agents based on one or more performance characteristics of agents. The method includes grading at least two agents on an optimal interaction and matching a caller with at least one of the two graded agents to increase the chance of the optimal interaction, e.g., matching a caller to the agent having the better performance. At theinitial block401, agents are graded on an optimal interaction, such as increasing revenue, decreasing costs, or increasing customer satisfaction. Grading can be accomplished by collating the performance of a contact center agent over a period of time on their ability to achieve an optimal interaction, such as a period of at least 10 days. However, the period of time can be as short as the immediately prior contact to a period extending as long as the agent's first interaction with a caller. Moreover, the method of grading agent can be as simple as ranking each agent on a scale of 1 to N for a particular optimal interaction, with N being the total number of agents. The method of grading can also comprise determining the average contact handle time of each agent to grade the agents on cost, determining the total or average sales revenue or number of sales generated by each agent to grade the agents on sales, or conducting customer surveys at the end of contacts with callers to grade the agents on customer satisfaction. The foregoing, however, are only examples of how agents may be graded; many other methods may be used.

At block402 a caller uses contact information, such as a telephone number or email address, to initiate a contact with the contact center. Atblock403, the caller is matched with an agent or group of agents such that the chance of an optimal interaction is increased, as opposed to a simple round robin matching method, for example. For instance, in a basic performance matching algorithm, an incoming caller is routed to the available agent having the highest sales rate.

The method may further include grading a group of at least two agents on two optimal interactions, weighting one optimal interaction against another optional interaction, and matching the caller with one of the two graded agents to increase the chance of a more heavily-weighted optimal interaction. In particular, agents may be graded on two or more optimal interactions, such as increasing revenue, decreasing costs, or increasing customer satisfaction, which may then be weighted against each other. The weighting can be as simple as assigning to each optimal interaction a percentage weight factor, with all such factors totaling to 100 percent. Any comparative weighting method can be used, however. The weightings placed on the various optimal interactions can take place in real-time in a manner controlled by the contact center, its clients, or in line with pre-determined rules.

FIG. 5 illustrate another exemplary model or method for matching a caller to an agent, and which may combine agent performance characteristics or grades, agent demographic data, agent psychographic data, and other business-relevant data about the agent (individually or collectively referred to in this application as “agent data”), along with demographic, psychographic, and other business-relevant data about callers (individually or collectively referred to in this application as “caller data”). Agent and caller demographic data can comprise any of: gender, race, age, education, accent, income, nationality, ethnicity, area code, zip code, marital status, job status, and credit score. Agent and caller psychographic data can comprise any of introversion, sociability, desire for financial success, and film and television preferences. It will be appreciated that the acts outlined in the flowchart ofFIG. 5 need not occur in that exact order.

This exemplary model or method includes determining at least one caller data for a caller, determining at least one agent data for each of two agents, using the agent data and the caller data in a pattern matching algorithm, and matching the caller to one of the two agents to increase the chance of an optimal interaction. At501, at least one caller data (such as a caller demographic or psychographic data) is determined. One way of accomplishing this is by retrieving this from available databases by using the caller's contact information as an index. Available databases include, but are not limited to, those that are publicly available, those that are commercially available, or those created by a contact center or a contact center client. In an outbound contact center environment, the caller's contact information is known beforehand. In an inbound contact center environment, the caller's contact information can be retrieved by examining the caller's CallerID information or by requesting this information of the caller at the outset of the contact, such as through entry of a caller account number or other caller-identifying information. Other business-relevant data such as historic purchase behavior, current level of satisfaction as a customer, or volunteered level of interest in a product may also be retrieved from available databases.

At502, at least one agent data for each of two agents is determined, which may include performance characteristic data as well as demographic or psychographic data. One method of determining agent demographic or psychographic data can involve surveying agents at the time of their employment or periodically throughout their employment. Such a survey process can be manual, such as through a paper or oral survey, or automated with the survey being conducted over a computer system, such as by deployment over a web-browser.

Though this advanced embodiment preferably uses agent performance characteristics or grades, demographic, psychographic, and other business-relevant data, along with caller demographic, psychographic, and other business-relevant data, other embodiments of the present invention can eliminate one or more types or categories of caller or agent data to minimize the computing power or storage necessary to employ the present invention.

Once agent data and caller data have been collected, this data is passed to a computational system. The computational system then, in turn, uses this data in a pattern matching algorithm at503 to create a computer model that matches each agent with the caller and estimates the probable outcome of each matching along a number of optimal interactions, such as the generation of a sale, the duration of contact, or the likelihood of generating an interaction that a customer finds satisfying.

The pattern matching algorithm to be used in the present invention can comprise any correlation algorithm, such as a neural network algorithm or a genetic algorithm. To generally train or otherwise refine the algorithm, actual contact results (as measured for an optimal interaction) are compared against the actual agent and caller data for each contact that occurred. The pattern matching algorithm can then learn, or improve its learning of, how matching certain callers with certain agents will change the chance of an optimal interaction. In this manner, the pattern matching algorithm can then be used to predict the chance of an optimal interaction in the context of matching a caller with a particular set of caller data, with an agent of a particular set of agent data. Preferably, the pattern matching algorithm is periodically refined as more actual data on caller interactions becomes available to it, such as periodically training the algorithm every night after a contact center has finished operating for the day.

At504, the pattern matching algorithm is used to create a computer model reflecting the predicted chances of an optimal interaction for each agent and caller matching. Preferably, the computer model will comprise the predicted chances for a set of optimal interactions for every agent that is logged in to the contact center as matched against every available caller. Alternatively, the computer model can comprise subsets of these, or sets containing the aforementioned sets. For example, instead of matching every agent logged into the contact center with every available caller, the example can match every available agent with every available caller, or even a narrower subset of agents or callers. Likewise, the exemplary method can match every agent that ever worked on a particular campaign—whether available or logged in or not—with every available caller. Similarly, the computer model can comprise predicted chances for one optimal interaction or a number of optimal interactions.

The computer model can also be further refined to comprise a suitability score for each matching of an agent and a caller. The suitability score can be determined by taking the chances of a set of optimal interactions as predicted by the pattern matching algorithm, and weighting those chances to place more or less emphasis on a particular optimal interaction as related to another optimal interaction. The suitability score can then be used to determine which agents should be connected to which callers.

III. Exemplary Agent Performance Estimation/Assigning Processes

According to one example, exemplary processes for estimating an agent's performance are provided, which may be used with a performance based matching or pattern matching processes.FIG. 6 illustrates an exemplary computer implemented method or process for estimating a performance characteristic for an agent. In this example, one or more agents of a plurality of agents are identified as having a number of calls fewer than a predefined threshold at602. The threshold number of calls may be determined in a variety of manners, including, for example, selecting a number based on a desired error rate, a percentage of the average number of calls per agent, number of converted calls, and so on.

The process then assigns a performance characteristic to the identified agents at604. The assigned performance characteristic may be an average for all agents in a call routing center or campaign. The assigned performance characteristic may further be some fraction lower or higher than an average, e.g., setting a conversion rate at 80% or 90% of the average conversion rate of a set of agents or for the particular campaign. Further, the assigned performance characteristic may include an adjustment to the particular agent's actual performance characteristic, e.g., adjusting an actual performance characteristic up or down toward an average performance characteristic for other agents.

The process further includes, at606, matching or routing a caller to an agent of a plurality of agents based, at least in part, on the performance characteristics of the agents, including the assigned performance characteristic(s) of those identified agents and actual performance characteristics of other agents. For example, in a simple performance based routing example, an incoming caller can be routed to the available agent having the highest (or most desired by the routing center) performance characteristic, whether estimated or actual. In other examples, the assigned performance characteristics are used as input into performance and/or pattern matching algorithms for routing callers to agents.

FIG. 7 illustrates another exemplary method or computer model for estimating an agent performance characteristic for use in routing callers to agents. This example includes determining errors in a particular performance characteristic and assigning agent performance characteristics based thereon. Initially, agent performance, p, of an agent who has made n calls resulting in h sales is given by:

$\begin{array}{cc}p=\frac{h}{n}& \left(1\right)\end{array}$
and the standard error, σ, of this estimate of p can be found from:

$\begin{array}{cc}\sigma =\sqrt{\frac{p\left(1-p\right)}{n}}& \left(2\right)\end{array}$

The fractional error, e, of the estimate of p can be defined as:

$\begin{array}{cc}e=\frac{\sigma }{p}& \left(3\right)\end{array}$

In one example, when the error becomes larger than a predefined threshold, t, the exemplary method and system provides an estimate of the agent's performance, different than provided by p. In one example, the system and method provides an estimate of (or adjusts) the agent's performance characteristic when the error equals or exceeds the threshold:

$\begin{array}{cc}e=\frac{\sigma }{p}=t& \left(4\right)\end{array}$

The number of calls an agent would have had to make to just meet the error threshold of t can be determined. For example, the number of calls to just meet the threshold indicated by N, and substituting (2) into (4) to eliminate σ:

$\begin{array}{cc}\frac{1}{p}\sqrt{\frac{p\left(1-p\right)}{N}}=t& \left(5\right)\end{array}$
which is equivalent to:

$\begin{array}{cc}N=\frac{1-p}{{\mathrm{pt}}^2}& \left(6\right)\end{array}$

Accordingly, if an agent has N (or more) calls the exemplary system will use the agents actual performance characteristic, e.g., as calculated in (1) above. If an agent has zero calls, the exemplary system and method assign the agent an average performance characteristic, e.g., the overall conversion rate across all agents, as a best estimate for his/her performance. In particular, and in one example, let the overall conversion rate R defined as:

$\begin{array}{cc}R=\frac{\text{Total number of sales across all agents}}{\text{Total number of calls across all agents}}& \left(7\right)\end{array}$

In some examples, rather than computing N from equation 6 above, a fixed N can be selected or set within the system, which may be based on the average conversion rate of the call center. For example, in a high CR environment (e.g., where 50% of calls result in a sale) N might be set to a small number, e.g., between 10 and 100. Whereas in a low CR system (e.g., where approximately 1% of calls results in a sale), N can be set higher, e.g., 200 or more.

In some examples, R can be defined to regress towards CR when agents do not have enough data, R, as the CR averaged across the whole call center. But it might be the case that new agents will in general perform less well than the call center average because they are inexperienced. Therefore, in one example, R can be set to a lower value than the call centre average, e.g., R=(overall conversion rate)/2, or some other lower number.

In one example, assigning an adjusted agent performance, p_adj, for agents whose number of calls taken falls in therange 0<n<N, the exemplary method assumes that as the number of calls approaches zero, the overall conversion rate is used, and as the number of calls approaches the threshold number N, the agent's own performance is used; in particular:

$\begin{array}{cc}\underset{n->0}{\lim }{p}_{\mathrm{adj}}=R\&\underset{n->N}{\lim }{p}_{\mathrm{adj}}=p& \left(8\right)\end{array}$

The exemplary method and system may interpolate between these two points, e.g., linearly interpolate between the two points. It will be recognized, of course, that other functions, e.g., a monotonic function, could also be used. In one example:
p_adj=mn+c   (9)

Using equation (9), and takingpoint 1 as the n=0 case, aspoint 2 as n=N, gives:
R=m×0+c (Point1)
p=m×N+c (Point2)   (10)

From which it follows that:

$\begin{array}{cc}m=\frac{p-R}{N}c=R& \left(11\right)\end{array}$

Therefore the calculation of adjusted agent performance, p_adj, for an agent with n calls, where 0<n<N, can be determined as follows:

$\begin{array}{cc}{p}_{\mathrm{adj}}=\frac{p-R}{N}n+R& \left(12\right)\end{array}$

With continued reference toFIG. 7, in one exemplary process, an average characteristic is determined for a plurality of agents at702. For example, with the illustrated example described here, R from equation (7) is computed. Further, agents are identified having an error, e.g., a fractional error, exceeding a predetermined threshold value at704. The agents may be identified by computing equations (1), (2), and (3), for example.

If the error exceeds the threshold, e.g., if e>t, a performance characteristic may be assigned to the identified agents at706. For example, N may be determined from equation (6) and used to determine an adjusted agent performance, p_adj, from (12). It is noted thatequation 12 provides an adjustment to an actual performance characteristic of the agent, the adjustment based on a liner interpolation between two points. In other examples, however, other interpolations may be used for adjusting/assigning a performance characteristic. Additionally, a substitution of a value unrelated to the agent's actual performance characteristic may be used, e.g., the average rate or some fraction of the average rate.

The final selection or mapping of a caller to an agent based on actual and assigned performance characteristics may then be passed to a routing engine or router for causing the caller to be routed to the agent at708. It is noted that the described actions of the exemplary methods described do not need to occur in the order in which they are stated and some acts may be performed in parallel. Further, additional matching models for scoring and mapping callers to agents may be used in a similar fashion, and a plurality of matching algorithms may be used and weighted against each other for determining a final selection of a caller-agent pair.

Determining a threshold number of calls or error value in practice may be evaluated and selected in many different manners For example, typically, one would like to set t such that the performance of a routing system is optimized. For example, using a small threshold may increase performance, however, if an unnecessarily small value is used, the system will needlessly reduce performance accuracy for agents who have a total number of calls <N and whose performance is far from the mean.

One manner for determining the effect of and selecting (or changing) a threshold value is with a Monte Carlo simulation.FIGS. 8A-8E illustrate graphs for an exemplary method and Monte Carlo simulation for selecting a desired threshold for estimating agent performance characteristics. In one example, an overall distribution of agent performance is chosen, which can be viewed as the true agent performances, i.e., with no error, as would be found for an infinite sample of calls for each agent.FIG. 8A illustrates an exemplary near normal distribution of 1000 agents with mean AP=0.3 and SD=0.1 in a Monte Carlo simulation (note that the handful of negative performances were set to zero).

For a given number of free agents (shown from the set {2, 5, 10, 20, 40}) and for a given value of t (shown from the set t=0, 0.1, 0.2, . . . , 1) free agents are selected randomly from the above distribution. The agent having the maximum true AP of the set of free agents is determined, and is therefore the correct agent for an exemplary performance based matching algorithm to select.

Next, each of the true agent performances is dithered (i.e., noise or error is intentionally added) to simulate the actually measured agent performances with an error due to the finite sample. For example, the dithering is applied by adding an error term to each agent's AP of N(0, 1):
p_Dithered=p_True(1+tN(0,1))   (13)
where p_Trueis the true agent performance, and N(μ, σ) is the normal distribution.

From this, one can check that the actually selected agent (i.e., one with maximum dithered agent performance) is the same as the one chosen in step (3), and if not, record the error in performance of the selected agent that occurred. This process can be repeated (e.g., 1,000 or more times) for each combination of number of free agents (e.g., 2, 5, 10, 20, and 40) and value of t.FIG. 8B illustrates the Monte Carlo results for this example; in particular, illustrating the percentage of calls routed to the agent with maximum true AP of the available agents versus t. As one would expect with at of zero, the correct (i.e., highest performing) agent is always chosen and the fraction of correct agents selection declines as t increases. Also, the loss of selection accuracy increases with the number of free agents available.

Another metric one can consider in determining a threshold value is the absolute value of the difference between the true agent performance of the actually selected agent (based on the noisy AP values) and the true agent performance of the agent that would have been selected (had the selection been based on true agent performances).FIG. 8C illustrates absolute value of error in performance of selected agent versus t (where the number of agents increases the mean absolute value of error for a given fraction error in agent performance, as shown).

Additionally, since the range of agent performances available varies between different mapping implementations, another metric includes expressing the error as a fraction of the standard deviation of the agent performance.FIG. 8D illustrates the error of AP of a selected agent measured in standard deviations of AP versus t. As illustrated, as the number of agents increases, the error in performance increases for a given t.

Accordingly, a contact center routing operator may analyze various different metrics in selecting a suitable or tolerable t, and which may further be varied depending on the number of agents, expected available agents, distribution of performance, and so on. Further, other estimation and simulation techniques may be used to assist an operator in setting thresholds.

FIG. 8E illustrates an exemplary impact on the increase (or boost) of a performance based matching process from the Monte Carlo simulation data; in particular, the fraction of boost lost versus t. If one assumes that boost is proportional to the average agent performances of the selected agents, one can compare the true AP's of the agents selected with the AP's of those which would have been selected had the true AP's been known, and thus calculate a fractional decrease in the boost. With this assumption it follows:

$\begin{array}{cc}\mathrm{Decrease}\mathrm{of}\mathrm{Boost}\propto \frac{\begin{array}{c}\text{Mean True AP of Agents}\\ \text{Selected from Dithered Data}\end{array}}{\begin{array}{c}\text{Mean True AP of Agents}\\ \text{Selected from True Data}\end{array}}& \left(14\right)\end{array}$

It is noted thatFIG. 8E, and the decrease in boost model, is deficient in that it assumes the same level of noise or error for all agents, whereas in a real call center there will be a range of imprecision depending on the accumulated number of calls for each agent. Accordingly, the model and calculations could easily be improved upon by including either an empirical or theoretical distribution of total calls per agent.

Many of the techniques described here may be implemented in hardware or software, or a combination of the two. Preferably, the techniques are implemented in computer programs executing on programmable computers that each includes a processor, a storage medium readable by the processor (including volatile and nonvolatile memory and/or storage elements), and suitable input and output devices. Program code is applied to data entered using an input device to perform the functions described and to generate output information. The output information is applied to one or more output devices. Moreover, each program is preferably implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language.

Each such computer program is preferably stored on a storage medium or device (e.g., CD-ROM, hard disk or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described. The system also may be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner.

FIG. 9 illustrates atypical computing system900 that may be employed to implement processing functionality in embodiments of the invention. Computing systems of this type may be used in clients and servers, for example. Those skilled in the relevant art will also recognize how to implement the invention using other computer systems or architectures.Computing system900 may represent, for example, a desktop, laptop or notebook computer, hand-held computing device (PDA, cell phone, palmtop, etc.), mainframe, server, client, or any other type of special or general purpose computing device as may be desirable or appropriate for a given application or environment.Computing system900 can include one or more processors, such as aprocessor904.Processor904 can be implemented using a general or special purpose processing engine such as, for example, a microprocessor, microcontroller or other control logic. In this example,processor904 is connected to abus902 or other communication medium.

Computing system900 can also include amain memory908, such as random access memory (RAM) or other dynamic memory, for storing information and instructions to be executed byprocessor904.Main memory908 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed byprocessor904.Computing system900 may likewise include a read only memory (“ROM”) or other static storage device coupled tobus902 for storing static information and instructions forprocessor904.

Thecomputing system900 may also includeinformation storage system910, which may include, for example, amedia drive912 and aremovable storage interface920. The media drive912 may include a drive or other mechanism to support fixed or removable storage media, such as a hard disk drive, a floppy disk drive, a magnetic tape drive, an optical disk drive, a CD or DVD drive (R or RW), or other removable or fixed media drive.Storage media918 may include, for example, a hard disk, floppy disk, magnetic tape, optical disk, CD or DVD, or other fixed or removable medium that is read by and written to bymedia drive912. As these examples illustrate, thestorage media918 may include a computer-readable storage medium having stored therein particular computer software or data.

In alternative embodiments,information storage system910 may include other similar components for allowing computer programs or other instructions or data to be loaded intocomputing system900. Such components may include, for example, aremovable storage unit922 and aninterface920, such as a program cartridge and cartridge interface, a removable memory (for example, a flash memory or other removable memory module) and memory slot, and otherremovable storage units922 andinterfaces920 that allow software and data to be transferred from theremovable storage unit918 tocomputing system900.

Computing system900 can also include acommunications interface924. Communications interface924 can be used to allow software and data to be transferred betweencomputing system900 and external devices. Examples ofcommunications interface924 can include a modem, a network interface (such as an Ethernet or other NIC card), a communications port (such as for example, a USB port), a PCMCIA slot and card, etc. Software and data transferred viacommunications interface924 are in the form of signals which can be electronic, electromagnetic, optical or other signals capable of being received bycommunications interface924. These signals are provided tocommunications interface924 via achannel928. Thischannel928 may carry signals and may be implemented using a wireless medium, wire or cable, fiber optics, or other communications medium. Some examples of a channel include a phone line, a cellular phone link, an RF link, a network interface, a local or wide area network, and other communications channels.

In this document, the terms “computer program product,” “computer-readable medium” and the like may be used generally to refer to physical, tangible media such as, for example,memory908,storage media918, orstorage unit922. These and other forms of computer-readable media may be involved in storing one or more instructions for use byprocessor904, to cause the processor to perform specified operations. Such instructions, generally referred to as “computer program code” (which may be grouped in the form of computer programs or other groupings), when executed, enable thecomputing system900 to perform features or functions of embodiments of the present invention. Note that the code may directly cause the processor to perform specified operations, be compiled to do so, and/or be combined with other software, hardware, and/or firmware elements (e.g., libraries for performing standard functions) to do so.

In an embodiment where the elements are implemented using software, the software may be stored in a computer-readable medium and loaded intocomputing system900 using, for example,removable storage media918, drive912 orcommunications interface924. The control logic (in this example, software instructions or computer program code), when executed by theprocessor904, causes theprocessor904 to perform the functions of the invention as described herein.

It will be appreciated that, for clarity purposes, the above description has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units, processors or domains may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controller. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality, rather than indicative of a strict logical or physical structure or organization.

The above-described embodiments of the present invention are merely meant to be illustrative and not limiting. Various changes and modifications may be made without departing from the invention in its broader aspects. The appended claims encompass such changes and modifications within the spirit and scope of the invention.

Claims (33)

We claim:

1. A computer implemented method for estimating agent performance in a call contact-center routing environment, the method comprising:

identifying, by one or more computers, one agent from a plurality of agents, wherein the one agent identified agent has a number of calls contact interactions that is fewer than a predetermined number of contact interactions;

computingand assigning, by the one or more computers, a respective assigned an outcome performance characteristic to for the one identified agent, wherein the assigned outcome performance is different from an actual outcome performance characteristic for the identified agent, and is based at least in part on the performance of one or more other of the agents a regression between a mean performance characteristic of the plurality of agents when the number of contact interactions of the one identified agent approaches zero and an actual performance characteristic of the one identified agent when the number of contact interactions of the one identified agent approaches the predetermined number of contact interactions;

assigning, by the one or more computers, the outcome performance characteristic to the one identified agent; and

routing, by the one or more computers, a caller contact to an agent of the plurality of agents based on respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the one identified agent.

2. The method ofclaim 1, wherein the assigned outcome performance characteristic is an average outcome performance characteristic of the plurality of agents.

3. The method ofclaim 1, wherein the assigned outcome performance characteristic is based at least in part on an average outcome performance characteristic of the plurality of agents.

4. The method ofclaim 1, wherein the assigned outcome performance characteristic for the one identified agent is based at least in part on agent outcome performance data of one or more agents having similar demographic data as the respective one identified agent.

5. The method ofclaim 1, wherein the computing and assigning step comprises computing, by the one or more computers, an adjustment to an the actual outcome performance characteristic of the one identified agent based on one or more criteria.

6. The method ofclaim 1, wherein the computing step comprises determining an interpolation between an the actual outcome performance characteristic value of the one identified agent and an outcome performance characteristic of the plurality of agents based at least in part on the criterion of a number of calls contact interactions received by the one identified agent.

7. The method ofclaim 1, wherein the predetermined number of calls contact interactions is determined relative to an average number of calls contact interactions for the plurality of agents.

8. The method ofclaim 1, wherein the predetermined number of calls contact interactions is associated with an error threshold in the an outcome performance characteristic of the plurality of agents.

9. The method ofclaim 1, wherein the agent outcome one identified agent's actual performance characteristic comprises a sales rate.

10. The method ofclaim 1, wherein routing the caller contact is based on an outcome a performance based matching algorithm.

11. The method ofclaim 1, wherein routing the caller contact is based on a pattern matching algorithm, and the assigned agent outcome performance characteristic is input into the pattern matching algorithm.

12. A computer implemented method for estimating agent outcome performance in a call contact-center routing environment, the method comprising:

determining, by one or more computers, respective actual outcome performance characteristics for each of a plurality of agents;

determining, by the one or more computers, an outcome performance characteristic representative of the plurality of agents;

identifying, by the one or more computers, a respective one of the plurality of agents having an error in the respective one identified agent's actual outcome performance characteristic greater than a predetermined threshold;

assigning, by the one or more computers, an outcome performance characteristic for the one identified agentwhich, wherein the assigned outcome performance characteristic is different from the actual outcome performance characteristic for the one identified agentand which, and wherein the assigned outcome performance characteristic is based at least in part on the representative outcome performance characteristic for representative of the plurality of agents; and

routing, by the one or more computers, a caller contact to one of the plurality of agents based on the respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the identified agent.

13. The method ofclaim 12, wherein the error comprises a fractional error.

14. The method ofclaim 12, wherein the assigned outcome performance characteristic is an average outcome performance characteristic of the plurality of agents.

15. The method ofclaim 12, wherein the assigned outcome performance characteristic is based at least in part on an average outcome performance characteristic of the plurality of agents.

16. The method ofclaim 12, wherein the assigning step comprises computing, by the one or more computers, an adjustment to an the actual outcome performance characteristic of the one identified agent.

17. The method ofclaim 16, wherein the computing step comprises determining an interpolation between an the actual outcome performance characteristic value of the respective one identified agent and an average outcome performance characteristic of the plurality of agents.

18. The method ofclaim 12, wherein the predetermined threshold is based on a fractional error of the one identified agent's actual outcome performance characteristic.

19. The method ofclaim 12, wherein the agent one identified agent's actual outcome performance characteristic comprises a sales rate.

20. The method ofclaim 12, wherein the routing the caller contact is based on an outcome a performance based matching algorithm.

21. The method ofclaim 12, wherein the routing the caller contact is based on a pattern matching algorithm, and the assigned agent outcome performance characteristic is input into the pattern matching algorithm.

22. A system for routing callers contacts to agents in a call contact-center routing environment, the system comprising:

one or more computers configured with computer-readable program code, that when executed, will cause performance of the steps:

identifying, by the one or more computers, one agent from a plurality of agents, wherein the one agent identified agent has a number of calls contact interactions that is fewer than a predetermined number of contact interactions;

computingand assigning, by the one or more computers, a respective an outcome performance characteristic to for the one identified agent, wherein the assigned outcome performance is different from an actual outcome performance characteristic for the respective identified agent, and is based at least in part on the performance of one or more other of the agents a regression between a mean performance characteristic of the plurality of agents when the number of contact interactions of the one identified agent approaches zero and an actual performance characteristic of the one identified agent when the number of contact interactions of the one identified agent approaches the predetermined number of contact interactions;

assigning, by the one or more computers, the outcome performance characteristic to the one assigned agent; and

routing, by the one or more computers, a caller contact to an agent of the plurality of agents based on respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the one identified agent.

23. A system for routing callers contacts to agents in a call contact-center routing environment, the system comprising:

one or more computers configured with computer-readable program code, that when executed, will cause performance of the steps:

determining, by the one or more computers, respective actual outcome performance characteristics for a plurality of agents;

determining, by the one or more computers, an outcome performance characteristic representative of the plurality of agents;

identifying, by the one or more computers, a respective one of the plurality of agents having an error in the respective one identified agent's actual outcome performance characteristic greater than a predetermined threshold;

assigning, by the one or more computers, an outcome performance characteristic for the identified agentwhich, wherein the assigned outcome performance characteristic is different from the actual outcome performance characteristic for the one identified agentand which, and wherein the assigned outcome performance characteristic is based at least in part on the representative outcome performance characteristic for representative of the plurality of agents; and

routing, by the one or more computers, a caller contact to one of the plurality of agents based on the respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the identified agent.

24. A non-transitory computer readable storage medium comprising computer readable instructions for carrying out, when executed by one or more computers, the a method of:

identifying, by the one or more computers, one agent from a plurality of agents, wherein the one agent identified agent has a number of calls contact interactions fewer than a predetermined number;

computingand assigning, by the one or more computers, a respective assigned an outcome performance characteristic to for the one identified agent, wherein the assigned outcome performance is different from an actual outcome performance characteristic for the identified agent, and is based at least in part on the performance of one or more other of the agents a regression between a mean performance characteristic of the plurality of agents when the number of contact interactions of the one identified agent approaches zero and an actual performance characteristic of the one identified agent when number of contact interactions of the one identified agent approaches the predetermined number of contact interactions;

assigning, by the one or more computers, the outcome performance characteristic to the one identified agent; and

routing, by the one or more computers, a caller contact to an agent of the plurality of agents based on respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the one identified agent.

25. A non-transitory computer readable storage medium comprising computer readable instructions for carrying out, when executed by one or more computers, the a method of:

determining, by the one or more computers, respective actual outcome performance characteristics for a plurality of agents;

determining, by the one or more computers, an outcome performance characteristic representative of the plurality of agents;

identifying, by the one or more computers, a respective one of the plurality of agents having an error in the respective one identified agent's actual outcome performance characteristic greater than a predetermined threshold;

assigning, by the one or more computers, an outcome performance characteristic for the one identified agentwhich, wherein the assigned outcome performance characteristic is different from the actual outcome performance characteristic for the one identified agentand which, and wherein the assigned outcome performance characteristic is based at least in part on the representative outcome performance characteristic for representative of the plurality of agents; and

routing, by the one or more computers, a caller contact to one of the plurality of agents based on the respective outcome performance characteristics of the plurality of agents, the respective outcome performance characteristics including the assigned outcome performance characteristicto the identified agent.

26. The method ofclaim 1, wherein the predetermined number of calls contact interactions is determined based at least in part on a conversion rate for the plurality of the agents.

27. The method ofclaim 1, further comprising:

computing, by the one or more computers, a boost of performance based at least in part on a representative number for agent outcome performances of the agents that were selected and based at least in part on the representative number for the agent outcome performances of the agents that would have been selected based on their actual agent performances.

28. The system ofclaim 22, wherein the predetermined number of calls contact interactions is determined based at least in part on a conversion rate for the plurality of the agents.

29. The system ofclaim 22, where wherein the one or more computers are further configured with the computer-readable program code, that, when executed, will cause performance of the step:

computing, by the one or more computers, a boost of performance based at least in part on a representative number for agent outcome performances of the agents that were selected and based at least in part on the representative number for the agent outcome performances of the agents that would have been selected based on their actual agent performances.

30. The system ofclaim 22, wherein the assigned outcome performance characteristic for the one identified agent is based at least in part on agent outcome performance data of one or more agents having similar demographic data as the respective one identified agent.

31. The system ofclaim 22, wherein the assigned outcome performance characteristic is based at least in part on an average outcome performance characteristic of the plurality of agents.

32. The system ofclaim 22, wherein the computing and assigning step comprises computing, by the one or more computers, an adjustment to an the actual outcome performance characteristic of the one identified agent based on one or more criteria.

33. The system of claim32 22, wherein the computing step comprises determining an interpolation between an the actual outcome performance characteristic value of the one identified agent and an outcome performance characteristic of the plurality of agents based at least in part on the criterion of a number of calls contact interactions received by the one identified agent.

Images