Simultaneous multithreading (SMT) is a technique for improving the overall efficiency ofsuperscalarCPUs withhardware multithreading. SMT permits multiple independentthreads of execution to better use the resources provided by modernprocessor architectures.
The termmultithreading is ambiguous, because not only can multiple threads be executed simultaneously on one CPU core, but also multiple tasks (with differentpage tables, differenttask state segments, differentprotection rings, differentI/O permissions, etc.). Although running on the same core, they are completely separated from each other.Multithreading is similar in concept topreemptive multitasking but is implemented at the thread level of execution in modern superscalar processors.
Simultaneous multithreading (SMT) is one of the two main implementations of multithreading, the other form beingtemporal multithreading (also known as super-threading). In temporal multithreading, only one thread of instructions can execute in any given pipeline stage at a time. In simultaneous multithreading, instructions from more than one thread can be executed in any given pipeline stage at a time. This is done without great changes to the basic processor architecture: the main additions needed are the ability to fetch instructions from multiple threads in a cycle, and a larger register file to hold data from multiple threads. The number of concurrent threads is decided by the chip designers. Two concurrent threads per CPU core are common, but some processors support many more.[1]
Because it inevitably increases conflict on shared resources, measuring or agreeing on its effectiveness can be difficult. However, measuredenergy efficiency of SMT with parallel native and managed workloads on historical 130 nm to 32 nm Intel SMT (hyper-threading) implementations found that in 45 nm and 32 nm implementations, SMT is extremely energy efficient, even with in-order Atom processors.[2] In modern systems, SMT effectively exploits concurrency with very little additional dynamic power. That is, even when performance gains are minimal the power consumption savings can be considerable.[2]
Some researchers[who?] have even shown that the extra threads can be used proactively to seed ashared resource like a cache, to improve the performance of another single thread, and claim this shows that SMT does not only increase efficiency. Others[who?] use SMT to provide redundant computation, for some level of error detection and recovery.[citation needed]
Nevertheless, in most current cases, SMT is about hidingstalls during high-latency activities such as memory access, therefore increasing efficiency as well as throughput of computations per amount of hardware used by making more use of existing resources.
In processor design, there are two ways to increase on-chip parallelism with fewer resource requirements: one is superscalar technique which tries to exploitinstruction-level parallelism (ILP); the other is multithreading approach exploitingthread-level parallelism (TLP).
Superscalar means executing multiple instructions at the same time while thread-level parallelism (TLP) executes instructions from multiple threads within one processor chip at the same time. There are many ways to support more than one thread within a chip, namely:
The key factor to distinguish them is to look at how many instructions the processor can issue in one cycle and how many threads from which the instructions come. For example, Sun Microsystems' UltraSPARC T1 is a multicore processor combined with fine-grain multithreading technique instead of simultaneous multithreading because each core can only issue one instruction at a time.
While multithreading CPUs have been around since the 1950s, simultaneous multithreading was first researched by IBM in 1968 as part of theACS-360 project.[3] The first major commercial microprocessor developed with SMT was theAlpha 21464 (EV8). This microprocessor was developed byDEC in coordination with Dean Tullsen of the University of California, San Diego, and Susan Eggers and Henry Levy of the University of Washington. The microprocessor was never released, since the Alpha line of microprocessors was discontinued shortly beforeHP acquiredCompaq which had in turn acquiredDEC. Dean Tullsen's work was also used to develop thehyper-threaded versions of the Intel Pentium 4 microprocessors, such as the "Northwood" and "Prescott".
TheIntelPentium 4 was the first modern desktop processor to implement simultaneous multithreading, starting from the 3.06 GHz model released in 2002, and since introduced into a number of their processors. Intel calls the functionalityHyper-Threading Technology, and provides a basic two-thread SMT engine. Intel claims up to a 30% speed improvement[4] compared against an otherwise identical, non-SMT Pentium 4. The performance improvement seen is very application-dependent; however, when running two programs that require full attention of the processor it can actually seem like one or both of the programs slows down slightly when Hyper-threading is turned on.[5] This is due to thereplay system of the Pentium 4 tying up valuable execution resources, increasing contention for resources such as bandwidth, caches,TLBs,re-order buffer entries, and equalizing the processor resources between the two programs which adds a varying amount of execution time. The Pentium 4 Prescott core gained a replay queue, which reduces execution time needed for the replay system. This was enough to completely overcome that performance hit.[6]
TheIntel Atom, first released in 2008, is the first Intel product to feature 2-way SMT (marketed as Hyper-Threading) without supporting instruction reordering, speculative execution, or register renaming. Intel reintroduced Hyper-Threading with theNehalem microarchitecture, after its absence on theCore microarchitecture.
IntelXeon Phi (2010–2020) has 4-way SMT (with time-multiplexed multithreading) with hardware-based threads which cannot be disabled, unlike regular Hyper-Threading.[7]
AMDBulldozer microarchitecture (2011) use two-thread "modules". In each module there are two separate integer cores but FlexFPU and L2 cache are shared, so it is only a partial SMT implementation.[8][9]
AMD'sZen family of microarchitectures has 2-way SMT. Most resources in aZen 5 core is competitively shared in SMT, allowing the active thread to take all resources (or "most" in the case of watermarked resources). The statically-partitioned exceptions are the micro-op queue, the retirement queue, and the FPU non-scheduling queue.[10]
The latest[when?]Imagination TechnologiesMIPS architecture designs include an SMT system known as "MIPS MT".[11] MIPS MT provides for both heavyweight virtual processing elements and lighter-weight hardware microthreads.RMI, a Cupertino-based startup, is the first MIPS vendor to provide a processorSOC based on eight cores, each of which runs four threads. The threads can be run in fine-grain mode where a different thread can be executed each cycle. The threads can also be assigned priorities.Imagination Technologies MIPS CPUs have two SMT threads per core.
IBM'sBlue Gene/Q has 4-way SMT.
The IBMPOWER5, announced in May 2004, comes as either a dual core dual-chip module (DCM), or quad-core or oct-core multi-chip module (MCM), with each core including a two-thread SMT engine.IBM's implementation is more sophisticated than the previous ones, because it can assign a different priority to the various threads, is more fine-grained, and the SMT engine can be turned on and off dynamically, to better execute those workloads where an SMT processor would not increase performance. This is IBM's second implementation of generally available hardware multithreading. In 2010, IBM released systems based on the POWER7 processor with eight cores with each having four Simultaneous Intelligent Threads. This switches the threading mode between one thread, two threads or four threads depending on the number of process threads being scheduled at the time. This optimizes the use of the core for minimum response time or maximum throughput. IBMPOWER8 has 8 intelligent simultaneous threads per core (SMT8).
IBM Z starting with thez13 processor in 2013 has two threads per core (SMT-2).
Although many people reported thatSun Microsystems' UltraSPARC T1 (known as "Niagara" until its 14 November 2005 release) and the now defunct processorcodenamed "Rock" (originally announced in 2005, but after many delays cancelled in 2010) are implementations ofSPARC focused almost entirely on exploiting SMT and CMP techniques, Niagara is not actually using SMT. Sun refers to these combined approaches as "CMT", and the overall concept as "Throughput Computing". The Niagara has eight cores, but each core has only one pipeline, so actually it uses fine-grained multithreading. Unlike SMT, where instructions from multiple threads share the issue window each cycle, the processor uses a round robin policy to issue instructions from the next active thread each cycle. This makes it more similar to abarrel processor. Sun Microsystems' Rock processor is different: it has more complex cores that have more than one pipeline.
TheOracle Corporation SPARC T3 has eight fine-grained threads per core; SPARC T4, SPARC T5, SPARC M5, M6 and M7 have eight fine-grained threads per core of which two can be executed simultaneously.
Fujitsu SPARC64 VI has coarse-grained Vertical Multithreading (VMT) SPARC VII and newer have 2-way SMT.
IntelItanium (IA-64, 2001–2020) Montecito uses coarse-grained multithreading and Tukwila and newer ones use 2-way SMT (with dual-domain multithreading).
VISC architecture (2016)[12][13][14][15] uses theVirtual Software Layer (translation layer) to dispatch a single thread of instructions to theGlobal Front End which splits instructions intovirtual hardware threadlets which are then dispatched to separate virtual cores. These virtual cores can then send them to the available resources on any of the physical cores. Multiple virtual cores can push threadlets into the reorder buffer of a single physical core, which can split partial instructions and data from multiple threadlets through the execution ports at the same time. Each virtual core keeps track of the position of the relative output. This form of multithreading can increase single threaded performance by allowing a single thread to use all resources of the CPU. The allocation of resources is dynamic on a near-single cycle latency level (1–4 cycles depending on the change in allocation depending on individual application needs. Therefore, if two virtual cores are competing for resources, there are appropriate algorithms in place to determine what resources are to be allocated where.
SMT introduces sharing of resources. Some sharing schemes (e.g. static division) do not allow one thread to take more of one resource even when another thread is not using it, creating a potential bottleneck for performance compared to the non-SMT case.[16] There is also a potential fairness issue. Newer SMT implementations try to minimize the occurrence of these problems with a combination of static partitioning, competitive sharing, and competitive sharing with watermarking, the latter being a combination of both.[17]
Critics argue that it is a considerable burden to put on software developers that they have to test whether simultaneous multithreading is good or bad for their application in various situations and insert extra logic to turn it off if it decreases performance. 2009 operating systems lack convenientAPI calls for this purpose and for preventing processes with different priority from taking resources from each other.[18] A cross-platformhwloc library is available to detect the presence of SMT as well asNUMA setups, both of which often require consideration from the programmer.Prime95 is one program that uses hwloc: by default it uses the additional SMT threads when doing integer trial-division, but only uses one thread per core for its usual floating-point-heavy operations.[19]
There is also a security concern with certain simultaneous multithreading implementations from bugs and side-channel information leaks. Intel's hyperthreading inNetBurst-based processors has a vulnerability through which it is possible for one application to steal acryptographic key from another application running in the same processor by monitoring its cache use.[20] There are also sophisticated machine learning exploits to HT implementation that were explained atBlack Hat 2018.[21]
| General | |
|---|---|
| Levels | |
| Multithreading | |
| Theory | |
| Elements | |
| Coordination | |
| Programming | |
| Hardware | |
| APIs | |
| Problems | |
