 | This articlemay rely excessively on sourcestoo closely associated with the subject, potentially preventing the article from beingverifiable andneutral. Please helpimprove it by replacing them with more appropriatecitations toreliable, independent sources.(February 2024) (Learn how and when to remove this message) |
Simultaneous and heterogeneous multithreading (SHMT) is asoftware framework that takes advantage ofheterogeneous computing systems that contain a mixture ofcentral processing units (CPUs),graphics processing units (GPUs), and special purposemachine learning hardware, for exampleTensor Processing Units (TPUs).[1][2]
Each component processes information differently. Often data has to move among processors, which can create bottlenecks, with one processor starving while waiting on another to finish.[1]
The system definesvirtual processors and virtual operations (VOPs). VOPs decompose into one or more high-level operations (HLOPs). It then distributes the operations across the processors. The runtime system then dynamically maps virtual processors to physical processors, assessing resource availability in order to keep all the processors busy. The scheduler employs a light-weight, quality-aware work-stealing (QAWS) policy.[1]
Conventional runtimes use assign one processor (set) to each subtask, leaving other types of processors idle. In other words, the CPU(s) run (possibly in parallel), then when that subtask completes, the next subtask is handed to the GPU(s). When they finish the next subtask is handed to the TPU(s).[2]
Adding software pipelining allows the second subtask to run using partial results from the first subtask, which improves resource utilization.[2]
SHMT takes things a step further, identifying subtasks that can run independently of others to the appropriate processor type, allow even better parallelism. Some subtasks can be performed on multiple processor types. SHMT can divide a single subtask across such processor types. Thus the fundamental breakthrough is to keep more processors working simultaneously, reducing time and energy costs.[2]
Researchers tested the concept using a typical smartphone configuration tweaked so that it resembled a data center server.[1]
The hardware wasNvidia's Jetson Nano module containing a quad-coreARM Cortex-A57 processor (CPU) and 128Maxwell architecture GPU cores. A GoogleEdge TPU was connected via itsM.2 Key E slot. The processors communicated via an onboardPCI Express (PCIe) interface. Shared data was hosted in a 4 GB 64-bitLPDDR4. The Edge TPU adds an 8 MB device memory.Ubuntu Linux 18.04 was the operating system.[1]
Compared to a conventional system performance increased by 1.95X boost, while energy consumption was reduced by 51%, on a range of benchmarks, includingBlack–Scholes, DCT8X8, DWT,FFT, Histogram, Hotspot,Laplacian, MF, Sobel, SRAD, and GMEAN.[1]
| General | |
|---|---|
| Levels | |
| Multithreading | |
| Theory | |
| Elements | |
| Coordination | |
| Programming | |
| Hardware | |
| APIs | |
| Problems | |
