US20070183331A1 - Method and system for providing indeterminate read data latency in a memory system - Google Patents

Abstract

A method and system for providing indeterminate read data latency in a memory system. The method includes determining if a local data packet has been received. If a local data packet has been received, then the local data packet is stored into a buffer device. The method also includes determining if the buffer device contains a data packet and determining if an upstream driver for transmitting data packets to a memory controller via an upstream channel is idle. If the buffer contains a data packet and the upstream driver is idle, then the data packet is transmitted to the upstream driver. The method further includes determining if an upstream data packet has been received. The upstream data packet is in a frame format that includes a frame start indicator and an identification tag for use by the memory controller in associating the upstream data packet with its corresponding read instruction. If an upstream data packet has been received and the upstream driver is not idle, then the upstream data packet is stored into the buffer device. If an upstream data packet has been received and the buffer device does not contain a data packet and the upstream driver is idle, then the upstream data packet is transmitted to the upstream driver. If the upstream driver is not idle, then any data packets in progress are continued being transmitted to the upstream driver.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This application is a continuation of U.S. patent application Ser. No. 11/289,193 filed Nov. 28, 2005, the contents of which are incorporated by reference herein in their entirety.
BACKGROUND OF THE INVENTION
• This invention relates to memory systems comprised of hub devices connected to a memory controller by a daisy chained channel. The hub devices are attached to, or reside upon, memory modules that contain memory devices. More particularly, this invention relates to the flow control of read data and the identification of read data returned to the controller by each hub device.
• Many high performance computing main memory systems use multiple fully buffered memory modules connected to a memory controller by one or more channels. The memory modules contain a hub device and multiple memory devices. The hub device fully buffers command, address and data signals between the memory controller and the memory devices. The flow of read data is controlled using either a leveled latency or position dependant latency technique. In both cases, the memory controller is able to predict the return time of read data requested from the memory modules and to schedule commands to avoid collisions as read data is merged onto the controller interface by each memory module.
• In some cases, the memory controller is able to issue a read data delay adder along with the read command. This instructs the targeted hub device to add additional delay to the return of read data in order to simplify the issuing of commands and to avoid collisions. In all cases, the read data must be returned in the order in which it was requested. Further, the total read data latency must be completely predictable by the memory controller. During run time operations, these two restrictions result in additional gaps being added to packets of read data that are returned from the memory modules. This adds latency to the average read operation. In addition, hubs are not able to use indeterminate techniques to return read data faster or slower than normal. These techniques include, but are not limited to, caching read data locally, reading memory devices speculatively, independently managing memory device address pages, data compression, etc.
• To optimize average read data latency under real workload conditions, and to enable advanced hub device capabilities, what is needed is a way to allow memory modules to return read data to the memory controller at an unpredicted time. This must be done in a way that does not corrupt read data and that allows the memory controller to identify each read data packet. Preventing data corruption by avoiding data collisions is especially complicated as hub devices merge local read data onto a cascaded memory controller channel.
BRIEF SUMMARY OF THE INVENTION
• Exemplary embodiments include a method for providing indeterminate read data latency. The method includes determining if a local data packet has been received. If a local data packet has been received, then the local data packet is stored into a buffer device. The method also includes determining if the buffer device contains a data packet and determining if an upstream driver for transmitting data packets to a memory controller via an upstream channel is idle. If the buffer contains a data packet and the upstream driver is idle, then the data packet is transmitted to the upstream driver. The method further includes determining if an upstream data packet has been received. The upstream data packet is in a frame format that includes a frame start indicator and an identification tag for use by the memory controller in associating the upstream data packet with its corresponding read instruction. If an upstream data packet has been received and the upstream driver is not idle, then the upstream data packet is stored into the buffer device. If an upstream data packet has been received and the buffer device does not contain a data packet and the upstream driver is idle, then the upstream data packet is transmitted to the upstream driver. If the upstream driver is not idle, then any data packets in progress are continued being transmitted to the upstream driver.
• Exemplary embodiments include a hub device in a memory system. The hub device includes a device for receiving data packets, an upstream driver for transmitting data packets to a memory controller via an upstream channel and a mechanism including instructions for facilitating indeterminate read data latency. The device for receiving data packets includes an upstream receiver for receiving upstream data packets from a downstream hub device and a memory interface for receiving local data packets from a local storage device. Each data packet is in a frame format that includes a frame start indicator and an identification tag for use by a memory controller in associating the data packet with its corresponding read instruction. The instructions on the mechanism facilitate determining if a local data packet has been received. If a local data packet has been received, then the local data packet is stored into a buffer device. The instructions also facilitate determining if the buffer device contains a data packet and determining if the upstream driver is idle. If the buffer contains a data packet and the upstream driver is idle, then the data packet is transmitted to the upstream driver. The instructions further facilitate determining if an upstream data packet has been received. If an upstream data packet has been received and the upstream driver is not idle, then the upstream data packet is stored into the buffer device. If an upstream data packet has been received and the buffer device does not contain a data packet and the upstream driver is idle, then the upstream data packet is transmitted to the upstream driver. If the upstream driver is not idle, then any data packets in progress are continued being transmitted to the upstream driver.
• Exemplary embodiments include a memory subsystem with one or more memory modules. The memory modules include one or more memory devices connected to a memory controller by a daisy chained channel. The read data is returned to the memory controller using a frame format that includes an identification tag and frame start indicator. The memory system also includes one or more hub devices on the memory modules for buffering address, commands and data. The hub devices include controller channel buffers that are used in conjunction with a preemptive local data merge algorithm to minimize read data latency and enable indeterminate read data return times to the memory controller.
• Further exemplary embodiments include a memory system with one or more memory modules. The memory modules include memory devices that are connected to a memory controller by a daisy chained channel. The read data is returned to the memory controller using a frame format that includes an identification tag and frame start indicator. The memory system also includes one or more hub devices connected to the memory modules for buffering address, commands and data. The hub devices include controller channel buffers that are used in conjunction with a preemptive local data merge algorithm to minimize read data latency and enable indeterminate read data return times to the memory controller.
BRIEF DESCRIPTION OF THE DRAWINGS
• Referring now to the drawings wherein like elements are numbered alike in the several FIGURES:
• FIG. 1 depicts an exemplary memory system with multiple levels of daisy chained memory modules with point-to-point connections;
• FIG. 2 depicts an exemplary memory system with hub devices that are connected to a memory modules and to a memory controller by a daisy chained channel;
• FIG. 3 depicts a hub logic device that may be utilized by exemplary embodiments;
• FIG. 4 is a exemplary process flow implemented by the hub logic device in exemplary embodiments; and
• FIG. 5 is a read data format that may be utilized by exemplary embodiments.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
• Exemplary embodiments utilize controller channel buffers (CCBs), read data frame formats with identification tags and a preemptive data merge technique to enable minimized and indeterminate read data latency. Exemplary embodiments allow memory modules to return read data to a memory controller at an unpredicted time. Identification tag information is added to the read data packet to indicate the read command that the data is a result of, as well as the hub where the data was read. The identification tag information is utilized by the controller to match the read data packet to the read commands issued by the controller. By using the identification tag information, read data can be returned in an order that is different from the issue order of the corresponding read commands.
• Exemplary embodiments also provide a preemptive data merge process to prevent data collisions on the upstream channel when implementing the indeterminate read data latency. A CCB is added to the hub device to temporarily store read data. When a memory device on the memory module reads data, the data is transferred from the memory interface to the buffer. When the hub device detects that an upstream data packet (i.e., a data packet being sent to the controller from a hub device that is downstream from the detecting hub device) is not in the middle of being transferred into the detecting hub device via an upstream channel (it typically takes several transfers to send the entire data packet), the detecting hub device checks to see if there is a read data packet in its CCB that is waiting to be sent upstream. If the hub device detects a read data packet in the CCB it drives the read data packet from the CCB onto the upstream data bus. In the meantime, if a new upstream data packet is received via the upstream data bus, the data packet is stored in the CCB on the hub device. In this manner, data packets coming upstream do not collide with data packets being sent upstream from the CCB on the hub device. In the case where there is more than one data packet in the CCB, a variety of methods may be implemented to determine which data packet to send next (e.g., the data packet from the oldest read command may be sent first).
• Exemplary embodiments apply to memory systems constructed of one ormore memory modules110 that are connected to amemory controller102 by a daisy chainedmemory channel114 as depicted inFIG. 1. Thememory modules110 contain both ahub device112 that buffers commands, address and data signals to and from thecontroller memory channel114 as well as one ormore memory devices108 connected to thehub device112. The downstream portion of thememory channel114, thedownstream channel104, transmits write data and memory operation commands to thehub devices112. The upstream portion of thecontroller channel114, theupstream channel106, returns requested read data (referred to herein as upstream data packets).
• FIG. 2 depicts an alternate exemplary embodiment that includes a memory system constructed of one ormore memory modules110 connected tohub devices112 that are further connected to amemory controller102 by a daisy chainedmemory channel114. In this embodiment, thehub device112 is not located on thememory module110; instead thehub device112 is in communication with thememory module110. As depicted inFIG. 2, thememory modules110 may be in communication with thehub devices112 via multi-drop connections and/or point-to-point connections. Other hardware configurations are possible, for example exemplary embodiments may utilize only a single level of daisy chainedhub devices112 and/ormemory modules110.
• FIG. 3 depicts ahub device112 withflow control logic308 utilized by exemplary embodiments to perform the processing described herein. Thehub device112 and the components within thehub device112 may be implemented in hardware and/or software. Thehub device112 receives upstream data packets on theupstream channel104 via the receiver logic304 (also referred to herein as an upstream receiver). The upstream data packets are data packets being sent to thecontroller102 from ahub device112 that is downstream from the receivinghub device112. An upstream data packet can be sent to the driver logic306 (also referred to herein as the upstream driver) to be driven towards thecontroller102 on theupstream channel106 or, if theupstream channel106 is busy, the upstream data packet can be temporarily stored in theCCB310 on thehub device112. The destination of the upstream data packet is determined by theflow control logic308 and implemented by sending a signal to thelocal data multiplexer312.
• In exemplary embodiments,CCBs310, or buffer devices, reside in thehub device112 and safely capture upstream data packet transfers (via the receiver logic304) that are shunted into theCCB310 while thehub device112 is merging its local data packets onto theupstream channel106. Local data packets are data packets that are read frommemory devices108 attached to thememory module110 being directed by thehub device112. Thesememory devices108 are also referred to herein as local storage devices. The data read from the local storage devices, the local data packets, are formatted for return on an upstream controller interface via the upstream driver and stored in theCCB310. The formatting includes serializing the local data packet into the proper frame format (e.g., see exemplary frame format depicted inFIG. 5), and inserting values into the identification tag (sourced from the read request), first transfer field, and bus cyclical redundancy code (CRC) field. In exemplary embodiments, the formatting of the local data packet is performed as part of storing the local data packet into theCCB310.
• When a data packet is received at thememory interface302, it is stored into theCCB310 while the local data packets are waiting to be merged onto the upstream channel106 (via the driver logic306). The identification tag within the data packet allows thememory controller102 to correlate a returned read data packet with its corresponding read data request command. The data packet also contains a small, easy to decode ‘start’, or first transfer (‘ft’) field (also referred to herein as a frame start indicator) delivered near the beginning of an upstream read data frame (data packets are formatted as read data frames) which indicates that a read data frame is present in the data packet. This is used by theflow control logic308 in thehub device112 to monitor the channel read data activity.
• When there is data in theCCBs310 from either a local read operation or from a previously shunted read data packet from a downstream hub device (the data packets in the CCB are referred to herein as stored data packets), thehub device112 will merge it onto theupstream channel106 via thedriver logic306 as soon as it is allowed. Thehub device112 merges local data onto theupstream channel106 whenever theupstream channel106 is idle, or immediately following the last transfer of a data packet that is currently in progress. Read data frames will never be bisected using this method, but read data frames that are in flight on theupstream channel106 that have not yet arrived at a hub device's112 local data multiplexer312 may be preempted and shunted into theCCB310. This allows gaps in read data on theupstream channel106 to be minimized which increases bus efficiency and results in reduced average read data latency under real world work load conditions.
• When there are multiple read data packets present in theCCBs310, thehub device112 can be configured to send the read data packet corresponding to the earliest read command. This minimizes undue latency on read requests issued tohub devices112 that are many daisy chain positions away from thememory controller102.Other CCB310 unload prioritization algorithms may also be implemented. For example, the identification tag field of the read data frame may contain a priority field. The priority field can be used to guide the unloading of theCCBs310. Alternatively, priority information may be delivered as the read data is requested.Hub devices112 can then compare the identification tag to previously recorded priority information to determine the location in theCCB310 to send next. A method may also be employed that occasionally sends lower priority data before high priority data to ensure that low priority data is not completely stalled by requests that have been tagged with a higher priority.
• FIG. 4 is a process flow that is facilitated by theflow control logic308 located in thehub device112 in exemplary embodiments. The process depicted inFIG. 4 performs preemptive local data merge and may be implemented by a mechanism including hardware and/or software instructions such as a finite state machine in theflow control logic308. The process starts atblock402 and is repeated, in exemplary embodiments, on a periodic basis (e.g., after each controller channel transfer, or upstream channel cycle). Atblock404 any local read data packets (i.e., frommemory devices108 onmemory modules110 attached to the hub device112) in thememory interface302 are loaded into theCCB310. This insures that theflow control logic308 is aware of and managing the upstream driving of local read data. Atblock406, it is determined if there is data in theCCB310. If there is no data in theCCB310, then the data is routed from thereceiver logic304 to thedriver logic306 atblock412. The routing is directed by theflow control logic308 by setting thelocal data multiplexer312 to send the upstream data packet to thedriver logic306 for driving the upstream data packet onto theupstream channel106 towards thecontroller102. Processing then continues at414, where processing is sent back to block404 at the next upstream channel cycle.
• If it is determined atblock406, that there is data in theCCB310 then block408 is performed to determine if an upstream channel operation is in process (i.e., is an upstream data packet or a local read data packet in the middle of being driven onto theupstream channel106 via the driver logic306). Processing continues atblock412 if an upstream channel operation is in process (i.e., the driver is busy). Atblock412, upstream read data packets are routed from thereceiver logic304 to thedriver logic306 by setting thelocal data multiplexer312 to send the upstream data packet to thedriver logic306. Alternatively, processing continues atblock410 if an upstream channel operation is not in process (i.e., the driver is idle) and there is data in theCCB310. Atblock410, data from theCCB310 is driven onto theupstream channel106 while any data packets received in thereceiver logic304 from theupstream channel106 are shunted (stored) into the nextavailable CCB310 location. The shunting is performed by theflow control logic308 directing the upstream data packets to be loaded into theCCB310. Processing then continues at414 which sends processing back to block404 at the next upstream channel cycle.
• FIG. 5 is an exemplary read data frame format for upstream data packets and local read data packets on theupstream channel106. The frame format depicted inFIG. 5 uses twenty-one signal lanes and each packet includes sixteen transfers. It includes a one bitfirst start indicator502 and anidentification tag504, as well as 256 bits (32 B) of readdata506 with abus CRCs508 for transmission error detection. Other combinations of signal lanes and transfer depths can be used to create frame formats that include a frame start indicator, read data identification tag and read data that are compatible with this invention.
• Exemplary embodiments pertain to a computer memory system constructed of daisy chained hub logic devices connected to, or contained upon, memory modules. The hubs are daisy chained on a memory controller channel and are further attached to memory devices on the memory modules. The memory controller issues requests for read data to the hubs which merge this read data from the memory modules onto the memory channel. Using channel buffers and packet identification tags, the hubs are able to return read data at a time unpredicted by the memory controller, and at a time that may preempt a read request that had been issued earlier, without loosing or corrupting any of the read data returned on the channel to the memory controller.
• Exemplary embodiments may be utilized to optimize average read data latency by more fully utilizing the upstream channel. Through the use of CCBs, read data frame formats with identification tags and a preemptive data merge technique, indeterminate read data latency may be performed to more fully utilize the controller channel.
• As described above, the embodiments of the invention may be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. Embodiments of the invention may also be embodied in the form of computer program code containing instructions embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.
• While the invention has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention, but that the invention will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another.

Claims (20)

1. A memory controller for use in a memory system, the memory controller comprising:

an upstream channel for receiving one or more read data packets at an unpredicted time from a downstream hub device, each packet in a frame format including a frame start indicator and an identification tag; and

computer instructions for correlating the received data packets with their corresponding read data request commands using the identification tags included in the read data packets.

2. The memory controller ofclaim 1 wherein each read data packet is received at an unpredicted time relative to its corresponding read request command.

3. The memory controller ofclaim 1 wherein the read data packets are received in an unpredicted order relative to an issue order of their corresponding read request commands.

4. The memory controller ofclaim 1 wherein the upstream channel is a daisy chain channel.

5. The memory controller ofclaim 1 wherein the hub device is physically located on a memory module.

6. The memory controller ofclaim 1 wherein the hub device includes a buffer device.

7. A method for providing indeterminate read data latency at a memory controller, the method comprising:

sending a memory data request from the memory controller to a memory module via a downstream bus, the memory module including a hub device;

receiving and processing the memory data request on the memory module;

sending a response data packet from the memory module to the memory controller via an upstream bus, the response data packet including a frame start indicator and an identification tag;

determining if the response data packet has been received at the memory controller using at least the frame start indicator; and

correlating the received response data packet with the memory data request based on the identification tag included in the response data packet.

8. The method ofclaim 7 wherein the identification tag in the memory response data packet is sourced from the read request.

9. The method ofclaim 7 wherein the receiving a response data packet at the memory controller occurs at an unpredicted time.

10. The method ofclaim 7 wherein the receiving of a response data packet at the memory controller occurs in an unpredicted order relative to an issue order of the corresponding memory data request command and other memory data request commands issued by the memory controller.

11. The method ofclaim 7 wherein the receiving and processing of the memory data request on the memory module includes:

receiving the memory data request at the memory module; and

servicing the memory data request at the memory module, resulting in the response data packet.

12. The method ofclaim 7 wherein the sending a response data packet includes executing a local preemptive data merge algorithm a the hub device to minimize read data latency and to enable indeterminate read data return times to the memory controller.

13. The method ofclaim 12 wherein the local preemptive data merge algorithm includes:

determining if a local data packet has been received;

if a local data packet has been received, then storing the local data packet into a buffer device;

determining if the buffer device contains a data packet;

determining if an upstream driver is idle;

if the buffer device contains a data packet and the upstream driver is idle, then transmitting the data packet to the upstream driver;

determining if an upstream data packet has been received;

if an upstream data packet has been received and the upstream driver is not idle, then storing the upstream data packet into the buffer device;

if an upstream data packet has been received and the buffer device does not contain a data packet and the upstream driver is idle, then transmitting the upstream data packet to the upstream driver; and

continuing to transmit any data packets in progress if the upstream driver is not idle.

14. The method ofclaim 13 wherein the determining if a local data packet has been received, the determining if the buffer device contains a data packet, the determining if an upstream driver is idle and the determining if a data packet has been received are performed on a periodic basis.

15. The method ofclaim 14 wherein the periodic basis is once every upstream channel cycle.

16. The method ofclaim 13 wherein the buffer device contains a plurality of data packets and the data packet is selected based on a prioritization algorithm.

17. The method ofclaim 16 wherein the prioritization algorithm selects the data packet based on the age of the read instruction that corresponds to the data packet.

18. The method ofclaim 17 wherein the prioritization algorithm selects the data packet based on a priority associated with the data packet.

19. The method ofclaim 1 wherein the frame format further includes a bus cyclical redundancy code (CRC) field.

20. A memory system comprising:

a memory controller including;

an upstream channel for receiving one or more read data packets at an unpredicted time; and

computer instructions for correlating the received read data packets with their corresponding read data request commands using identification tags included in the read data packets;

one or more memory modules with one or more memory devices connected to the memory controller by a daisy chained channel, wherein the read data is returned to the memory controller as the read data packets using a frame format that includes the identification tag and a frame start indicator; and

one or more hub devices on the memory modules for buffering address, commands and data, the hub devices including controller channel buffers used in conjunction with a preemptive local data merge algorithm to minimize read data latency and enable indeterminate read data return times to the memory controller.

Images