Glossary#

Homogeneous Memory System

A system that operates on a single type of memory implemented using a singletechnology.

Heterogeneous Memory System

A system that operates on multiple types of memories, possibly implementedusing different technologies, often managed by different entities.

Memory Tiering

An organization of different types of memory storage within a system, eachhaving distinct characteristics, performance, and cost attributes. Thesememory tiers are typically organized in a hierarchy, with faster, moreexpensive memory located closer to the processor and slower, less expensivememory located further away.

Memory Access Initiator

A component in a computer system that initiates or requests access to thecomputer’s memory subsystem. This could be a CPU, GPU, or other I/O and cachedevices.

Memory Target

Any part of the memory subsystem that can handle memory access requests. Thiscould be the OS-accessible main memory (RAM), video memory that resides onthe graphics cards, memory caches, storage, external memory devices connectedusing CXL.mem protocol, etc.

Memory Page

A fixed-length contiguous block of virtual memory, described by a singleentry in the page table. It is the smallest unit of data for memorymanagement in a virtual memory operating system.

Enlightened Application

An application that explicitly manages data allocation distribution amongdifferent types of memory and handles data migration between them.

Unenlightened Application

An application that relies on the underlying infrastructure (OS, frameworks,libraries) that offers various memory tiering and migration solutions withoutany code modifications.

Memory Pool

A memory management technique used in computer programming and softwaredevelopment, where relatively large blocks of memory are preallocated usingmemory provider and then passed to a pool allocator for fine-grainmanagement. The pool allocator could divide these blocks into smaller chunksand use them for application allocations depending on its needs. Typicallypool allocators focus on the low fragmentation and constant allocation time,so they are used to optimize memory allocation and deallocation in scenarioswhere efficiency and performance are critical.

Pool Allocator

A memory allocator type used to efficiently manage memory pools. Among theexisting ones are jemalloc or oneTBB’s Scalable Memory Allocator.

Memory Provider

A software component responsible for supplying memory or managing memorytargets. A single memory provider can efficiently manage the memoryoperations for one or multiple devices within the system or other memorysources like file-backed or user-provided memory. Memory providers areresponsible for coarse-grain allocations and management of memory pages.

High Bandwidth Memory (HBM)

A high-speed computer memory. It is used in conjunction with high-performancegraphics accelerators, network devices, and high-performance data centers, ason-package cache in CPUs, FPGAs, supercomputers, etc.

Compute Express Link (CXL)

An open standard for high-speed, high-capacity central processing unit(CPU)-to-device and CPU-to-memory connections, designed for high-performancedata center computers. CXL is built on the serial PCI Express (PCIe) physicaland electrical interface and includes PCIe-based block input/output protocol(CXL.io), cache-coherent protocols for accessing system memory (CXL.cache),and device memory (CXL.mem).

oneAPI Threading Building Blocks (oneTBB)

A C++ template library developed by Intel for parallel programming onmulti-core processors. TBB broke down the computation into tasks that can runin parallel. The library manages and schedules threads to execute these tasks.

jemalloc

A general-purpose malloc implementation that emphasizes fragmentationavoidance and scalable concurrency support. It provides introspection, memorymanagement, and tuning features functionalities.Jemalloc uses separatepools (“arenas”) for each CPU which avoids lock contention problems inmultithreading applications and makes them scale linearly with the number ofthreads.

Unified Shared Memory (USM)

A programming model which provides a single memory address space that isshared between CPUs, GPUs, and possibly other accelerators. It simplifiesmemory management by transparently handling data migration between the CPUand the accelerator device as needed.