Non-uniform memory access (NUMA) is acomputer memory design used inmultiprocessing, where the memory access time depends on the memory location relative to the processor. Under NUMA, a processor can access its ownlocal memory faster than non-local memory (memory local to another processor or memory shared between processors).[1] NUMA is beneficial for workloads with high memorylocality of reference and lowlock contention, because a processor may operate on a subset of memory mostly or entirely within its own cache node, reducing traffic on the memory bus.[2]
NUMA architectures logically follow in scaling fromsymmetric multiprocessing (SMP) architectures. They were developed commercially during the 1990s byUnisys,Convex Computer (laterHewlett-Packard),Honeywell Information Systems Italy (HISI) (laterGroupe Bull),Silicon Graphics (laterSilicon Graphics International),Sequent Computer Systems (laterIBM),Data General (laterEMC, nowDell Technologies),Digital (laterCompaq, thenHP, nowHPE) andICL. Techniques developed by these companies later featured in a variety ofUnix-likeoperating systems, and to an extent inWindows NT.
The first commercial implementation of a NUMA-based Unix system was[where?][when?] the Symmetrical Multi Processing XPS-100 family of servers, designed by Dan Gielan of VAST Corporation forHoneywell Information Systems Italy.
Modern CPUs operate considerably faster than the main memory they use. In the early days of computing and data processing, the CPU generally ran slower than its own memory. The performance lines of processors and memory crossed in the 1960s with the advent of the firstsupercomputers. Since then, CPUs increasingly have found themselves "starved for data" and forced to stall to wait for data to arrive from memory (e.g. forVon-Neumann architecture-based computers, seeVon Neumann bottleneck). Many supercomputer designs of the 1980s and 1990s focused on providing high-speed memory access as opposed to faster processors, allowing the computers to work on large data sets at speeds other systems could not approach.
Limiting the number of memory accesses provided the key to extracting high performance from a modern computer. For commodity processors, this meant installing an ever-increasing amount of high-speedcache memory and using increasingly sophisticated algorithms to avoidcache misses. But the dramatic increase in size of both the operating systems and the applications run on them has generally overwhelmed these cache-processing improvements. Multi-processor systems without NUMA make the problem considerably worse. Now a system can starve several processors at the same time, notably because only one processor can access the computer's memory at a time.[3]
NUMA attempts to address this problem by providing separate memory for each processor, avoiding the performance hit when several processors attempt to address the same memory. For problems involving spread data (common forservers and similar applications), NUMA can improve the performance over a single shared memory by a factor of roughly the number of processors (or separate memory banks).[4] Another approach to addressing this problem is themulti-channel memory architecture, in which a linear increase in the number of memory channels increases the memory access concurrency linearly.[5]
Of course, not all data ends up confined to a single task, which means that more than one processor may require the same data. To handle these cases, NUMA systems include additional hardware or software to move data between memory banks. This operation slows the processors attached to those banks, so the overall speed increase due to NUMA heavily depends on the nature of the running tasks.[4]
AMD implemented NUMA with itsOpteron processor (2003), usingHyperTransport.Intel announced NUMA compatibility for its x86 andItanium servers in late 2007 with itsNehalem andTukwila CPUs.[6] Both Intel CPU families share a commonchipset; the interconnection is called IntelQuickPath Interconnect (QPI), which provides extremely high bandwidth to enable high on-board scalability and was replaced by a new version called IntelUltraPath Interconnect with the release ofSkylake (2017).[7]
Nearly all CPU architectures use a small amount of very fast non-shared memory known ascache to exploitlocality of reference in memory accesses. With NUMA, maintainingcache coherence across shared memory has a significant overhead. Although simpler to design and build, non-cache-coherent NUMA systems become prohibitively complex to program in the standardvon Neumann architecture programming model.[8]
Typically, ccNUMA uses inter-processor communication between cache controllers to keep a consistent memory image when more than one cache stores the same memory location. For this reason, ccNUMA may perform poorly when multiple processors attempt to access the same memory area in rapid succession. Support for NUMA inoperating systems attempts to reduce the frequency of this kind of access by allocating processors and memory in NUMA-friendly ways and by avoiding scheduling and locking algorithms that make NUMA-unfriendly accesses necessary.[9]
Alternatively, cache coherency protocols such as theMESIF protocol attempt to reduce the communication required to maintain cache coherency.Scalable Coherent Interface (SCI) is anIEEE standard defining a directory-based cache coherency protocol to avoid scalability limitations found in earlier multiprocessor systems. For example, SCI is used as the basis for the NumaConnect technology.[10][11]
One can view NUMA as a tightly coupled form ofcluster computing. The addition ofvirtual memory paging to a cluster architecture can allow the implementation of NUMA entirely in software. However, the inter-node latency of software-based NUMA remains several orders of magnitude greater (slower) than that of hardware-based NUMA.[2]
Since NUMA largely influences memory access performance, certain software optimizations are needed to allow scheduling threads and processes close to their in-memory data.
As of 2011,[update] ccNUMA systems are multiprocessor systems based on theAMD Opteron processor, which can be implemented without external logic, and the IntelItanium processor, which requires the chipset to support NUMA. Examples of ccNUMA-enabled chipsets are the SGI Shub (Super hub), the Intel E8870, theHP sx2000 (used in the Integrity and Superdome servers), and those found in NEC Itanium-based systems. Earlier ccNUMA systems such as those fromSilicon Graphics were based onMIPS processors and theDECAlpha 21364 (EV7) processor.
| General | |
|---|---|
| Levels | |
| Multithreading | |
| Theory | |
| Elements | |
| Coordination | |
| Programming | |
| Hardware | |
| APIs | |
| Problems | |
