US9270602B1 - Transmit rate pacing of large network traffic bursts to reduce jitter, buffer overrun, wasted bandwidth, and retransmissions - Google Patents

Abstract

A system, method and medium is disclosed which includes selecting, at a software component of a network traffic management device, a first bucket having a first predetermined transmit time. The disclosure includes populating one or more selected data packet descriptors associated with one or more corresponding data packets in the first bucket. The disclosure includes releasing the first bucket to a hardware component of the network traffic management device, wherein the hardware component processes the one or more data packet descriptors of the first bucket for the first predetermined transmit time.

Description

FIELD

This technology relates to a system and method for pacing network traffic between network devices.

BACKGROUND

It is very common for a modern server to transmit large blocks of data in one burst to a single destination where the network path to the destination has much lower bandwidth than the really large bandwidth of the server's path within the data center and the first few hops along that path. Often these bursts are in response to a request for data, and the data is read from a storage medium (e.g., a disk) in a large block to help amortize the cost of the read operation. The large chunk of data is then dumped by the server's OS into the network at full speed, adding latency for other traffic following some portion of the same path through the network. This latency is unnecessary since other traffic could easily have been interleaved with the packets of the burst if the packets of the burst were spaced in time to match the true bandwidth of the full path to the destination. Further, such bursts can result in loss of some of the burst data due to overruns in of the buffering in the network path to the destination. Such losses necessitate retransmission of some of the large chunk of data—effectively reducing the gains that were hoped to be achieved by the batching of the read operation into a large chunk and reducing the overall throughput of the server. If the packets in these bursts were, instead, spread out in time at a pace matching the full network path data rate, both of these problems are easily solved

What is needed is a system and method which overcomes these disadvantages.

SUMMARY

In an aspect, a method for pacing data packets from one or more sessions comprises selecting, at a software component of a network traffic management device, a first bucket having a first predetermined transmit time. The method comprises populating one or more selected data packet descriptors associated with one or more corresponding data packets in the first bucket. The method comprises releasing the first bucket to a hardware component of the network traffic management device, wherein the hardware component processes the one or more data packet descriptors of the first bucket for the first predetermined transmit time.

In an aspect, a processor readable medium having stored thereon instructions for pacing data packets from one or more sessions, comprising machine executable code which when executed by at least one processor and/or network interface a network traffic management device to perform a method comprising selecting a first bucket having a first predetermined transmit time; populating one or more selected data packet descriptors associated with one or more corresponding data packets in the first bucket; releasing the first bucket to a hardware component of the network traffic management device, wherein the hardware component processes the one or more data packet descriptors of the first bucket for the first predetermined transmit time.

In an aspect, a network traffic management device comprises a memory containing non-transitory machine readable medium comprising machine executable code having stored thereon instructions for pacing data packets from one or more sessions. A network interface configured to communicate with one or more servers over a network. A processor coupled to the network interface and the memory, the processor configured to execute the code which causes the processor to perform, with the network interface, a method comprising: selecting a first bucket having a first predetermined transmit time; populating one or more selected data packet descriptors associated with one or more corresponding data packets in the first bucket; releasing the first bucket to a hardware component of the network traffic management device, wherein the hardware component processes the one or more data packet descriptors of the first bucket for the first predetermined transmit time.

In one or more of the above aspects, the method performs comprises selecting, at the software component, a second bucket having a second predetermined transmit time that is the same as the first transmit time of the first bucket; populating one or more selected data packet descriptors associated with one or more corresponding data packets in the second bucket; releasing the second bucket to the hardware component of the network traffic management device, wherein the hardware component processes the one or more data packet descriptors of the second bucket for the second predetermined transmit time quanta.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example system environment that includes a network traffic management device in accordance with an aspect of the present disclosure.

FIG. 2A is a block diagram of the network traffic management device in accordance with an aspect of the present disclosure.

FIG. 2B illustrates a block diagram of the network interface in accordance with an aspect of the present disclosure.

FIG. 2C illustrates further details of the network traffic management device in accordance with an aspect of the present disclosure.

FIG. 3 illustrates an example transmit ring or bucket in accordance with an aspect of the present disclosure.

FIG. 4 illustrates a time line of an example transmit ring or bucket in accordance with an aspect of the present disclosure.

FIG. 5 illustrates a block diagram of hardware based time enforcement in performed by a high speed bridge (HSB) priority mechanism in accordance with an aspect of the present disclosure.

FIG. 6 illustrates an example transmit time line in accordance with an aspect of the present disclosure.

FIG. 7 illustrates a flow chart of the software implementation of populating buckets with data packet descriptors in accordance with an aspect of the present disclosure.

FIG. 8 illustrates a flow chart of the software implementation of populating buckets with data packet descriptors in accordance with an aspect of the present disclosure.

While these examples are susceptible of embodiment in many different forms, there is shown in the drawings and will herein be described in detail preferred examples with the understanding that the present disclosure is to be considered as an exemplification and is not intended to limit the broad aspect to the embodiments illustrated.

DETAILED DESCRIPTION

FIG. 1 is a diagram of an example system environment that includes a network traffic management device in accordance with an aspect of the present disclosure. Theexample system environment100 includes one or more Web and/or non Web application servers102 (referred generally as “servers”), one ormore client devices106 and one or more networktraffic management devices110, although theenvironment100 can include other numbers and types of devices in other arrangements. The networktraffic management device110 is coupled to theservers102 via local area network (LAN)104 andclient devices106 via awide area network108. Generally, client device requests are sent over thenetwork108 to theservers102 which are received or intercepted by the networktraffic management device110.

Client devices106 comprise network computing devices capable of connecting to other network computing devices, such as networktraffic management device110 and/orservers102. Such connections are performed over wired and/or wireless networks, such asnetwork108, to send and receive data, such as for Web-based requests, receiving server responses to requests and/or performing other tasks. Non-limiting and non-exhausting examples ofsuch client devices106 include personal computers (e.g., desktops, laptops), tablets, smart televisions, video game devices, mobile and/or smart phones and the like. In an example,client devices106 can run one or more Web browsers that provide an interface for operators, such as human users, to interact with for making requests for resources to different web server-based applications and/or Web pages via thenetwork108, although other server resources may be requested by client devices.

Theservers102 comprise one or more server network devices or machines capable of operating one or more Web-based and/or non Web-based applications that may be accessed by other network devices (e.g. client devices, network traffic management devices) in theenvironment100. Theservers102 can provide web objects and other data representing requested resources, such as particular Web page(s), image(s) of physical objects, JavaScript and any other objects, that are responsive to the client devices' requests. It should be noted that theservers102 may perform other tasks and provide other types of resources. It should be noted that while only twoservers102 are shown in theenvironment100 depicted inFIG. 1B, other numbers and types of servers may be utilized in theenvironment100. It is contemplated that one or more of theservers102 may comprise a cluster of servers managed by one or more networktraffic management devices110. In one or more aspects, theservers102 may be configured implement to execute any version of Microsoft® IIS server, RADIUS server, DIAMETER server and/or Apache® server, although other types of servers may be used.

Network108 comprises a publicly accessible network, such as the Internet, which is connected to theservers102,client devices106, and networktraffic management devices110. However, it is contemplated that thenetwork108 may comprise other types of private and public networks that include other devices. Communications, such as requests fromclients106 and responses fromservers102, take place over thenetwork108 according to standard network protocols, such as the HTTP, UDP and/or TCP/IP protocols, as well as other protocols. As per TCP/IP protocols, requests from the requestingclient devices106 may be sent as one or more streams of data packets overnetwork108 to the networktraffic management device110 and/or theservers102. Such protocols can be utilized by theclient devices106, networktraffic management device110 and theservers102 to establish connections, send and receive data for existing connections, and the like.

Further, it should be appreciated thatnetwork108 may include local area networks (LANs), wide area networks (WANs), direct connections and any combination thereof, as well as other types and numbers of network types. On an interconnected set of LANs or other networks, including those based on differing architectures and protocols. Network devices such as client devices,106,servers102, networktraffic management devices110, routers, switches, hubs, gateways, bridges, cell towers and other intermediate network devices may act within and between LANs and other networks to enable messages and other data to be sent between network devices. Also, communication links within and between LANs and other networks typically include twisted wire pair (e.g., Ethernet), coaxial cable, analog telephone lines, full or fractional dedicated digital lines including T1, T2, T3, and T4, Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines (DSLs), wireless links including satellite links and other communications links known to those skilled in the relevant arts. Thus, thenetwork108 is configured to handle any communication method by which data may travel between network devices.

LAN104 comprises a private local area network that allows communications between the one or more networktraffic management devices110 and one ormore servers102 in the secured network. It is contemplated, however, that the LAN104 may comprise other types of private and public networks with other devices. Networks, including local area networks, besides being understood by those skilled in the relevant arts, have already been generally described above in connection withnetwork108 and thus will not be described further.

As shown in theexample environment100 depicted inFIG. 1B, the one or more networktraffic management devices110 is interposed betweenclient devices106 with which it communicates with vianetwork108 andservers102 with which it communicates with via LAN104. In particular to the present disclosure, the networktraffic management device110 operates in conjunction with a clustered multi-processing (CMP) system which includes one or more networktraffic management devices110, each of which having one or more cores or processors200 (FIG. 2A). Generally, the networktraffic management device110 manages network communications, which may include one or more client requests and server responses, via thenetwork108 between theclient devices106 and one or more of theservers102. In any case, the networktraffic management device110 may manage the network communications by performing several network traffic related functions involving the communications. Some functions include, but are not limited to, load balancing, access control, and validating HTTP requests using JavaScript code that are sent back to requestingclient devices106.

FIG. 2A is a block diagram of the network traffic management device in accordance with an aspect of the present disclosure. As shown inFIG. 2A, the example networktraffic management device110 includes one or more device processors orcores200, one or more device I/O interfaces202, one ormore network interfaces204, and one ormore device memories206, which are coupled together by one ormore bus208. It should be noted that the networktraffic management device110 can be configured to include other types and/or numbers of components and is thus not limited to the configuration shown inFIG. 2A.

Device processor200 of the networktraffic management device110 comprises one or more microprocessors configured to execute computer/machine readable and executable instructions stored in thedevice memory206. Such instructions, when executed by one ormore processors200, implement general and specific functions of the networktraffic management device110, including the inventive process described in more detail below. It is understood that theprocessor200 may comprise other types and/or combinations of processors, such as digital signal processors, micro-controllers, application specific integrated circuits (“ASICs”), programmable logic devices (“PLDs”), field programmable logic devices (“FPLDs”), field programmable gate arrays (“FPGAs”), and the like. Theprocessor200 is programmed or configured according to the teachings as described and illustrated herein.

Device I/O interfaces202 comprise one or more user input and output device interface mechanisms. The interface may include a computer keyboard, mouse, display device, and the corresponding physical ports and underlying supporting hardware and software to enable the networktraffic management device110 to communicate with other network devices in theenvironment100. Such communications may include accepting user data input and providing user output, although other types and numbers of user input and output devices may be used. Additionally or alternatively, as will be described in connection withnetwork interface204 below, the networktraffic management device110 may communicate with the outside environment for certain types of operations (e.g. smart load balancing) via one or more network management ports.

Network interface204 comprises one or more mechanisms that enable the networktraffic management device110 to engage in network communications over the LAN104 and thenetwork108 using one or more of a number of protocols, such as TCP/IP, HTTP, UDP, RADIUS and DNS. However, it is contemplated that thenetwork interface204 may be constructed for use with other communication protocols and types of networks.Network interface204 is sometimes referred to as a transceiver, transceiving device, or network interface card (NIC), which transmits and receives network data packets over one or more networks, such as the LAN104 and thenetwork108. In an example, where the networktraffic management device110 includes more than one device processor200 (or aprocessor200 has more than one core), each processor200 (and/or core) may use the samesingle network interface204 or a plurality of network interfaces204. Further, thenetwork interface204 may include one or more physical ports, such as Ethernet ports, to couple the networktraffic management device110 with other network devices, such asservers102. Moreover, theinterface204 may include certain physical ports dedicated to receiving and/or transmitting certain types of network data, such as device management related data for configuring the networktraffic management device110 or client request/server response related data.

Bus208 may comprise one or more internal device component communication buses, links, bridges and supporting components, such as bus controllers and/or arbiters. Thebus208 enables the various components of the networktraffic management device110, such as theprocessor200, device I/O interfaces202,network interface204, anddevice memory206, to communicate with one another. However, it is contemplated that thebus208 may enable one or more components of the networktraffic management device110 to communicate with one or more components in other network devices as well. Example buses include HyperTransport, PCI, PCI Express, InfiniBand, USB, Firewire, Serial ATA (SATA), SCSI, IDE and AGP buses. However, it is contemplated that other types and numbers of buses may be used, whereby the particular types and arrangement of buses will depend on the particular configuration of the networktraffic management device110.

Device memory206 comprises computer readable media, namely computer readable or processor readable storage media, which are examples of machine-readable storage media. Computer readable storage/machine-readable storage media may include volatile, nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information. Examples of computer readable storage media include RAM, BIOS, ROM, EEPROM, flash/firmware memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the information, which can be accessed by a computing or specially programmed network device, such as the networktraffic management device110.

Such storage media includes computer readable/processor-executable instructions, data structures, program modules, or other data, which may be obtained and/or executed by one or more processors, such asdevice processor200. Such instructions, when executed, allow or cause theprocessor200 to perform actions, including performing the inventive processes described below. Thememory206 may contain other instructions relating to the implementation and operation of an operating system for controlling the general operation and other tasks performed by the networktraffic management device110.

FIG. 2B illustrates a block diagram of the network interface in accordance with an aspect of the present disclosure. In particular,FIG. 2B shows the DMA processes used bynetwork interface204 for using multiple independent DMA channels with corresponding multiple applications, where each application has its own driver, and for sending packets.

As illustrated inFIG. 2B, thehost system111 can send a network data packet stored inhost memory212 to thenetwork108 vianetwork interface controller204 andEthernet port236. A send DMA operation is performed when thehost system111 uses a DMA channel to move a block of data from host memory22 to a network interface controller peripheral (not shown) vianetwork108. To perform a send DMA operation, thehost processor200 places the target network data packet intoDMA packet buffer216 and creates a DMA send descriptor (not shown separately) in send DMA descriptor rings218. The DMA send descriptor is jointly managed by thehost system111 and thenetwork interface controller204. The DMA send descriptor includes an address field and length field. The address field points to the start of the target network data packet inDMA packet buffer216. The length field declares how many bytes of target data are present in theDMA packet buffer216. The DMA send descriptor also has a set of bit flags (not shown) used to signal additional target data control and status information.

By way of example only, return DMA descriptor rings and send DMA descriptor rings218 can be physically in the same hardware memory blocks functioning as return and send DMA rings, respectively, at different times. Alternatively, separate and distinct memory blocks withinhost memory212DMA memory resources214 may be reserved for each return DMA descriptor rings and send DMA descriptor rings218.

Host system111 places the send descriptor on the send DMA descriptor rings218 inhost system memory212. Thehost processor200 determines the QoS of the network packet to be transferred to thenetwork108 and moves the network packet to the appropriateDMA packet buffer216 and places the descriptor on the appropriate descriptor rings 1-4 in send DMA descriptor rings218. The descriptor ring in send DMA descriptor rings218 is chosen by thehost system111 which selects the DMA channel, its associated peripheral, and the QoS level within the DMA channel. Send descriptors created byhost system111 in send DMA descriptor rings218 can be of variable types, where each descriptor type can have a different format and size. The send DMA descriptor rings218 are capable of holding descriptors of variable type.

Thehost processor200 writes one or more mailbox registers230 of thenetwork interface controller204 to notify thenetwork interface controller204 that the packet is ready. In performing this notification, thehost processor200 performs a write operation to a memory mapped network interface controller register (mailbox register230). Thehost processor200 can report the addition of multiple descriptors onto the send DMA ring in a single update, or alternatively, in multiple updates.

The appropriate packet DMA engine withinDMA engine224 is notified that the packet is ready. Thepacket DMA engine224 can be selected from available DMA channels, or if a specific application has a dedicated DMA channel, the associatedpacket DMA engine224 for that channel is used. TheDMA engine224 retrieves the DMA descriptor from the send DMA descriptor rings218. When multiple descriptors are outstanding in the send DMA descriptor rings218, theDMA Engine224 may retrieve more than one descriptor. Retrieving multiple descriptors at a time maximizes bus bandwidth and hardware efficiency. TheDMA engine224 is capable of receiving and processing send descriptors of variable type, format, and size.

As outlined above, thepacket DMA engine224 monitors the progress of the host DMA operations via a set of mailbox registers230. Eachpacket DMA engine224 supports its own set of mailbox registers230. The mailbox registers230 reside in a mapped address space of thenetwork interface controller204. When appropriate, thehost processor200 accesses the mailbox registers230 by performing memory mapped read and write transactions to the appropriate target address. The mailbox registers230 also contain ring status information for the Ring toQoS Mapper228. In this send DMA example, thepacket DMA engine224 reads the send descriptor, performs the DMA operation defined by it, and reports to thehost system111 that the DMA operation is complete. During the DMA operation, data is received from one or more CPU Bus read transactions (e.g., HyperTransport or PCI Express read transactions).

DMA scheduler226 chooses packets out ofpacket buffers216 based upon the priority of the queued network data packets and schedules the transfer to the appropriatepacket DMA engine224. For clarity and brevity, only a single packet buffer, a single DMA scheduler, and DMA engine are shown inFIG. 2B, but it should be understood that additional packet buffers, DMA schedulers, and DMA engines supporting the independent DMA channels 1-n and associated applications App(1)-App(n) can be included innetwork interface controller204.

The packet buffers216 are selected based on the novel scheme (discussed below) usingDMA scheduler226. TheDMA scheduler226 selects which descriptor ring 1-4 out of return DMA descriptor rings (also referred to as return DMA rings, or send rings) withinDMA memory resources212 to service and the matchingpacket buffer216 is accessed for a single packet. The scheduling process is then repeated for the next packet.

Each network packet retrieved from apacket buffer216 is routed to the appropriate DMA channel controlled by the respective packet DMA engine such as thepacket DMA engine224. The DMA channel segments the network packet for delivery to hostmemory212 via several, smaller, HyperTransport packets. These HyperTransport packets are interleaved with HyperTransport packets from the other DMA channels in thenetwork interface controller204.

Ring toQoS Mapper228 examines the assigned send DMA ring in send DMA descriptor rings218 and receives packet data and packet control information from thepacket DMA engine224. Using the control information, the Ring toQoS Mapper228 stamps the appropriate QoS onto the network data packet, thereby allowinghost system111 to send the network data packet back to thenetwork108. For example, using the control information, the Ring toQoS Mapper228 can create and prepend a HiGig header to the packet data.

An egressDMA routing interface232 arbitrates access to the network for DMA send packets. When a Ring toQoS Mapper228 has a network packet ready to send, the egressDMA routing interface232 arbitrates its access to theEthernet port236 and routes the packet to the correct interface if there is more than one present in thenetwork interface controller204. The egressDMA routing interface232 behaves like a crossbar switch and monitors its attached interfaces for available packets. When a packet becomes available, the egressDMA routing interface232 reads the packet from the selected ring toQoS mapper228 and writes it to the destination interface. The egressDMA routing interface232 moves complete packets toEthernet MACs234. When multiple sources are contending for egressDMA routing interface232, the egressDMA routing interface232 uses a weighted arbitration scheme as discussed in more detail below.

Thenetwork interface controller204 provides DMA services to a host complex such as thehost system111 on behalf of its attached I/O devices such as theEthernet port236. DMA operations involve the movement of data between thehost memory212 and thenetwork interface controller204. Thenetwork interface controller204 creates and manages HyperTransport or other types of CPU Bus read/write transactions targeting host memory22. Data transfer sizes supported by DMA channels maintained by various components ofapplication delivery controller110 are much larger than the maximum HyperTransport or CPU bus transaction size. Thenetwork interface controller204 segments single DMA operations into multiple smaller CPU Bus or HyperTransport transactions. Additionally, thenetwork interface controller204 creates additional CPU bus or HyperTransport transactions to support the transfer of data structures between thenetwork interface controller204 andhost memory212.

FIG. 2C illustrates further details of the network traffic management device in accordance with an aspect of the present disclosure. In particular, the networktraffic management device110 is shown handling a plurality of independent applications App(1)-App(n) being executed by one or more processors (e.g., host processor200) inhost system111. Each application in the plurality of applications App(1)-App(n) has its own respective application driver shown asDriver 1,Driver 2, . . . , Driver ‘n’ associated with the respective application, where the index n denotes an unlimited number of executing applications and drivers. Applications App(1)-App(n) send and receive data packets from and to the network108 (and/or LAN104), respectively, using respective DMA channels (e.g., DMA channels 1-n). DMA channels 1-n are uniquely assigned to individual applications out of App(1)-App(n). In this example, drivers 1-n manage access to respective DMA channels 1-n and do not require knowledge of each other or a common management database or entity (e.g., a hypervisor). By way of example only, each of applications App(1)-App(n) can be independent instances of different applications, or alternatively, may be independent instances of the same application, or further, may be different operating systems supported by different processors in host system111 (e.g., host processor200).

DMA channels 1-n each have unique independent resources allotted to them, for example, a unique PCI bus identity including a configuration space and base address registers, an independent view ofhost system memory212, a unique set of DMA descriptor ring buffers, a unique set ofpacket buffers216, unique DMA request/completion signaling (through interrupts or polled memory structures), and other resources. Each of DMA channels 1-n is unique and independent thereby permitting management by separate unique drivers 1-n.

Thenetwork interface controller204 classifies received packets to determine destination application selected from applications App(1)-App(n) and thereby selects the matching DMA channel to deliver the packet to the corresponding application. By way of example only, packet classification includes reading packet header fields thereby permitting application identification. Further by way of example only, packet classification includes hash calculation for distribution of packets across multiple instances of the same application, and/or reading a cookie stored, for example, in thenetwork interface controller204 associated with the application and the received network packet.

In general, the present disclosure utilizes hardware and software in the network traffic management device when pacing network traffic being sent to another network device (e.g. client, server). The purpose is for the network traffic management device to pace delivery of session data to the client to match the rate at which the client consumes the data, which adds value for mobile clients and networks. The network traffic management device utilizes a transmit time calendar having a plurality of quanta of transmit times. For example, one quanta may be 1 microsecond, although other time durations are contemplated.

FIG. 3 illustrates an example transmit ring or bucket in accordance with an aspect of the present disclosure. As shown inFIG. 3, thebucket300 is a software based data construct which holds a plurality of data packet DMA descriptors which point to the pending transmit packets and has packets from a plurality of various, different sessions. Thebucket300 has afence302 which marks the last descriptor in the bucket. Software determines what data packets to send and when to send those data packets. The software communicates with the hardware component wherein the software enhances the data structures to communicate a time component with regard to when the selected data packets are to be sent. In particular, the software component divides session data into multiple maximum segment sized (MSS) or smaller sized packets. The software component distributes the packets in buckets separated by fixed transmit times, wherein the buckets are then handed off to the hardware component, such as a DMA ring, to handle the delivery of the data packets populated in the buckets, by processing the buckets themselves.

FIG. 4 illustrates a time line of an example transmit ring or bucket in accordance with an aspect of the present disclosure. As can be seen inFIG. 4, the transmit time calendar utilizes a plurality ofbuckets300 wheresession data400, such as data packet DMA descriptors is populated into thebuckets300. Each bucket has a predetermined size and adjacent buckets are separate by a transmit time or quanta that is fixed (shown as T1-T4). It should be noted the time calendar is configured to have a finite number of buckets, wherein the calendar effectively wraps around to the first bucket after the last bucket has been handled. In addition, multiple packet descriptors can be placed in the same bucket to increase bandwidth. The software component may decide whether to pace a data packet, as opposed to being bulk data (FIG. 5) based on size, type of traffic the data packet is associated with, QoS parameters (based on flow) and the like.

In an aspect, one ormore buckets300 can be skipped for distribution, by the software component, to increase intra-packet spacing. One way for determining how many buckets to skip before placing a data packet descriptor in abucket300 can be based on the type or class of application in which the data is delivered (e.g. video streaming). For example, video streaming applications would benefit from a constant rate of data delivery. Another way of determining (based on speed) would be using TCP congestion control algorithms which calculate how many packets are to be sent over time. The software does not have to hunt for holes where packet descriptors can be inserted. Instead, the software component need only to drop packet descriptors into the one ormore buckets300.

Once abucket300 is populated by the software component, the software component releases the bucket to the hardware component, such as thenetwork interface204. In particular, the bucket contents are written into the network interface's204DMA ring218 en-mass. In an aspect, the hardware component can determine whether the size of a particular data packet in a bucket is of a threshold size such that the data packet can be processed within the allotted time quanta.

A gross timer is applied by the hardware component in writing the packets into theDMA ring218, wherein the poll loop time of theprocessor200 of the networktraffic management device110 is used as the gross timer. The sum of released bucket time quanta's must excel poll loop time.

The fence304 marking at the end of aparticular bucket300 serves as a timing boundary at the end of the bucket to allow the hardware component to do precise bucket to bucket timing.

FIG. 5 illustrates a block diagram of hardware based time enforcement in performed by a high speed bridge (HSB) priority mechanism in accordance with an aspect of the present disclosure. As shown inFIG. 5, data packets that are written to buckets that are released to the hardware are received in apacing send ring500. In contrast, non-paced data (data not written to buckets) are received in abulk send ring502.

In an aspect, the pacing sendring500 is given higher priority in terms of being sent to thering arbitrator506. Thus, the bulk traffic in the bulk sendringer502 advances to thearbitrator506 when the paced traffic (from the pacing send ring500) is blocked or is absent. Apacing timer504 in coupled to the pacing sendring500 and thearbitrator506, wherein the pacing timer is reset at the start of anew bucket300. Thepacing timer504 also blocks traffic at bucket boundaries until the bucket quanta time expires.

FIG. 6 illustrates an example transmit time line in accordance with an aspect of the present disclosure. As shown in thetime line600, the hardware component consumes the data packets in thefirst bucket602 for the entire time quanta which ends at boundary A. In thesecond bucket604, it is shown that the hardware component finishes writing the paced data packets such that additional time remains for the allotted time. The hardware component thereafter processes bulk data for the remaining amount of time in the time quanta until reaching boundary B. The same occurs inbucket606. Inbucket608, there is a period of time where no data is written, which can result in no bulk data or paced data being available for writing to the DMA engine.

FIG. 7 illustrates a flow chart of the software implementation of populating buckets with data packet descriptors in accordance with an aspect of the present disclosure. As shown inFIG. 7, theprocess700 occurs when a first bucket of a plurality of buckets is selected by the application module210 of the network traffic management device (Block702). The application module210 thereafter populates the first bucket with one or more data packet descriptors associated with data packets to be released to the network interface204 (Block704). The application module210 thereafter determines whether there are additional paced or bulk data packets that can be added to the current selected bucket (Block706). If so, the process repeats back to Block704. If not, the process proceeds to Block708.

Once the application module210 has populated the bucket with the last data packet, the application module210 applies a fence which represents the last data packet for that bucket (Block708). The application module210 thereafter determines if the selected bucket is the last bucket in the time calendar (Block710). If so, the application module210 effectively wraps around the time calendar and selects the first bucket (Block702). If not, the application module210 selects the next bucket in the time calendar (Block712), wherein the process proceeds back to Block704.

FIG. 8 illustrates a flow chart of the software implementation of populating buckets with data packet descriptors in accordance with an aspect of the present disclosure. As shown inFIG. 8, theprocess800 occurs when a first bucket of a plurality of buckets is selected by thenetwork interface204 of the network traffic management device (Block802). Thenetwork interface204 thereafter processes paced data packet(s) for the selected bucket (Block804). Thenetwork interface204 thereafter determines whether additional time is available for the selected bucket (Block806). If so, thenetwork interface204 determines if bulk data packets are present in the bucket (Block808). If so, thenetwork interface204 processes the bulk data for the bucket (Block810). The process then repeats back toBlock806.

If no additional allotted time is left, thenetwork interface204 selects thenext bucket812 and the process proceeds back to Block804. Referring back toBlock808, if no bulk data is available for processing for a particular bucket, thenetwork interface204 does not write any data packets and allows the remaining time for the bucket quanta to expire (Block814). The process then proceeds to Block812.

Having thus described the basic concepts, it will be rather apparent to those skilled in the art that the foregoing detailed disclosure is intended to be presented by way of example only, and is not limiting. Various alterations, improvements, and modifications will occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested hereby, and are within the spirit and scope of the examples. Additionally, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed system and/or processes to any order except as may be specified in the claims. Accordingly, the system and method is limited only by the following claims and equivalents thereto.

Claims (18)

What is claimed is:

1. A method for transmitting data packets at an optimized rate, the method comprising:

populating, by the network traffic management computing device, a plurality of buckets with one or more selected data packet descriptors associated with one or more corresponding ones of a subset of a plurality of data packets to be transmitted as paced and another subset of the data packets to be transmitted as bulk;

releasing, by the network traffic management computing device, one of the buckets to a hardware component comprising a pacing send ring and a bulk send ring, the releasing comprising writing one or more of the subset of the data packets to the pacing send ring and one or more of the another subset of the data packets to the bulk send ring; and

transmitting, by the network traffic management computing device, the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring for a predetermined transmit time, wherein the subset of the data packets are strictly prioritized over the another subset of the data packets and the one or more of the another subset of the data packets are only transmitted within the predetermined transmit time when the pacing send ring is emptied during the predetermined transmit time.

2. The method as set forth inclaim 1, further comprising repeating, by the network traffic management computing device, the releasing and transmitting for each other of the buckets.

3. The method as set forth inclaim 1, wherein the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring are transmitted by a direct memory access (DMA) transmit engine.

4. The method as set forth inclaim 1, wherein the subset of the data packets to be transmitted is identified as paced and the another subset of the data packets is identified as bulk based on a data packet size, an associated type of traffic, or a quality of service (QoS) parameter.

5. The method as set forth inclaim 1, further comprising:

determining, by the network traffic management computing device, when another one of the buckets should be skipped based on a type of application from which the ones of the data packets associated with the selected data packet descriptors populated therein originated; and

skipping, by the network traffic management computing device, the another one of the buckets such that the releasing and transmitting are not repeated for the another one of the buckets, when the determining indicates that the another one of the buckets should be skipped.

6. The method as set forth inclaim 1, further comprising waiting, by the network traffic management computing device, to release another one of the buckets when the one or more of the subset of the data packets are transmitted from the pacing send ring and the one or more of the another subset of the data packets are transmitted from the bulk send ring prior to the expiration of the predetermined transmit time.

7. A non-transitory computer readable medium having stored thereon instructions for transmitting data packets at an optimized rate, comprising executable code which when executed by at least one processor and/or network interface causes the processor and/or network interface to perform steps comprising:

populating a plurality of buckets with one or more selected data packet descriptors associated with one or more corresponding ones of a subset of a plurality of data packets to be transmitted as paced and another subset of the data packets to be transmitted as bulk;

releasing one of the buckets to a hardware component comprising a pacing send ring and a bulk send ring, the releasing comprising writing one or more of the subset of the data packets to the pacing send ring and one or more of the another subset of the data packets to the bulk send ring; and

transmitting the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring for a predetermined transmit time, wherein the subset of the data packets are strictly prioritized over the another subset of the data packets and the one or more of the another subset of the data packets are only transmitted within the predetermined transmit time when the pacing send ring is emptied during the predetermined transmit time.

8. The non-transitory computer readable medium as set forth inclaim 7, wherein the executable code when executed by the processor and/or the network interface further causes the processor and/or the network interface to perform at least one additional step comprising repeating the releasing and transmitting for each other of the buckets.

9. The non-transitory computer readable medium as set forth inclaim 7, wherein the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring are transmitted by a direct memory access (DMA) transmit engine.

10. The non-transitory computer readable medium as set forth inclaim 7, wherein the subset of the data packets to be transmitted is identified as paced and the another subset of the data packets is identified as bulk based on a data packet size, an associated type of traffic, or a quality of service (QoS) parameter.

11. The non-transitory computer readable medium as set forth inclaim 7, wherein the executable code when executed by the processor and/or the network interface further causes the processor and/or the network interface to perform at least one additional step comprising:

determining when another one of the buckets should be skipped based on a type of application from which the ones of the data packets associated with the selected data packet descriptors populated therein originated; and

skipping the another one of the buckets such that the releasing and transmitting are not repeated for the another one of the buckets, when the determining indicates that the another one of the buckets should be skipped.

12. The non-transitory computer readable medium as set forth inclaim 7, wherein the executable code when executed by the processor and/or the network interface further causes the processor and/or the network interface to perform at least one additional step comprising waiting to release another one of the buckets when the one or more of the subset of the data packets are transmitted from the pacing send ring and the one or more of the another subset of the data packets are transmitted from the bulk send ring prior to the expiration of the predetermined transmit time.

13. A network traffic management computing device comprising at least one processor and/or network interface and a memory coupled to the processor and/or network interface which is configured to be capable of executing programmed instructions comprising and stored in the memory to:

populate a plurality of buckets with one or more selected data packet descriptors associated with one or more corresponding ones of a subset of a plurality of data packets to be transmitted as paced and another subset of the data packets to be transmitted as bulk;

release one of the buckets to a hardware component comprising a pacing send ring and a bulk send ring, the releasing comprising writing one or more of the subset of the data packets to the pacing send ring and one or more of the another subset of the data packets to the bulk send ring; and

transmit the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring for a predetermined transmit time, wherein the subset of the data packets are strictly prioritized over the another subset of the data packets and the one or more of the another subset of the data packets are only transmitted within the predetermined transmit time when the pacing send ring is emptied during the predetermined transmit time.

14. The network traffic management computing device as set forth inclaim 13, wherein the processor and/or network interface coupled to the memory is further configured to be capable of executing at least one additional programmed instruction comprising and stored in the memory to repeat the releasing and transmitting for each other of the buckets.

15. The network traffic management computing device as set forth inclaim 13, wherein the one or more of the subset of the data packets from the pacing send ring and the one or more of the another subset of the data packets from the bulk send ring are transmitted by a direct memory access (DMA) transmit engine.

16. The network traffic management computing device as set forth inclaim 13, wherein the subset of the data packets to be transmitted is identified as paced and the another subset of the data packets is identified as bulk based on a data packet size, an associated type of traffic, or a quality of service (QoS) parameter.

17. The network traffic management computing device as set forth inclaim 13, wherein the processor and/or network interface coupled to the memory is further configured to be capable of executing at least one additional programmed instruction comprising and stored in the memory to:

determine when another one of the buckets should be skipped based on a type of application from which the ones of the data packets associated with the selected data packet descriptors populated therein originated; and

skip the another one of the buckets such that the releasing and transmitting are not repeated for the another one of the buckets, when the determining indicates that the another one of the buckets should be skipped.

18. The network traffic management computing device as set forth inclaim 13, wherein the processor and/or network interface coupled to the memory is further configured to be capable of executing at least one additional programmed instruction comprising and stored in the memory to wait to release another one of the buckets when the one or more of the subset of the data packets are transmitted from the pacing send ring and the one or more of the another subset of the data packets are transmitted from the bulk send ring prior to the expiration of the predetermined transmit time.

Images