Kernel BOs¶

A kernel BO is created as part of driver load (e.g. uC firmware images, GuCADS, etc...) or a BO created as part of a user operation which requiresa kernel BO (e.g. engine state, memory for page tables, etc...). These BOsare typically mapped in the GGTT (any kernel BOs aside memory for page tablesare in the GGTT), are pinned (can’t move or be evicted at runtime), have avmap (Xe can access the memory via xe_map layer) and have contiguous physicalmemory.

More details of why kernel BOs are pinned and contiguous below.

User BOs¶

A user BO is created via the DRM_IOCTL_XE_GEM_CREATE IOCTL. Once it iscreated the BO can be mmap’d (via DRM_IOCTL_XE_GEM_MMAP_OFFSET) for useraccess and it can be bound for GPU access (via DRM_IOCTL_XE_VM_BIND). Alluser BOs are evictable and user BOs are never pinned by Xe. The allocation ofthe backing store can be deferred from creation time until first use which iseither mmap, bind, or pagefault.

Runtime eviction¶

Runtime evictions refers to during normal operations where TTM decides itneeds to move a BO. Typically this is because TTM needs to make room foranother BO and the evicted BO is first BO on LRU list that is not locked.

An example of this is a new BO which can only be placed in VRAM but there isnot space in VRAM. There could be multiple BOs which have sysmem and VRAMplacement rules which currently reside in VRAM, TTM trigger a will move ofone (or multiple) of these BO(s) until there is room in VRAM to place the newBO. The evicted BO(s) are valid but still need new bindings before the BOused again (exec or compute mode rebind worker).

Another example would be, TTM can’t find a BO to evict which has anothervalid placement. In this case TTM will evict one (or multiple) unlocked BO(s)to a temporary unreachable (invalid) placement. The evicted BO(s) are invalidand before next use need to be moved to a valid placement and rebound.

In both cases, moves of these BOs are scheduled behind the fences in the BO’sdma-resv slots.

WW locking tries to ensures if 2 VMs use 51% of the memory forward progressis made on both VMs.

Runtime eviction uses per a GT migration engine (TODO: link to migrationengine doc) to do a GPU memcpy from one location to another.

Rebinds after runtime eviction¶

When BOs are moved, every mapping (VMA) of the BO needs to rebound beforethe BO is used again. Every VMA is added to an evicted list of its VM whenthe BO is moved. This is safe because of the VM locking structure (TODO: linkto VM locking doc). On the next use of a VM (exec or compute mode rebindworker) the evicted VMA list is checked and rebinds are triggered. In thecase of faulting VM, the rebind is done in the page fault handler.

Suspend / resume eviction of VRAM¶

During device suspend / resume VRAM may lose power which means the contentsof VRAM’s memory is blown away. Thus BOs present in VRAM at the time ofsuspend must be moved to sysmem in order for their contents to be saved.

A simple TTM call (ttm_resource_manager_evict_all) can move all non-pinned(user) BOs to sysmem. External BOs that are pinned need to be manuallyevicted with a simple loop + xe_bo_evict call. It gets a little trickierwith kernel BOs.

Some kernel BOs are used by the GT migration engine to do moves, thus wecan’t move all of the BOs via the GT migration engine. For simplity, use aTTM memcpy (CPU) to move any kernel (pinned) BO on either suspend or resume.

Some kernel BOs need to be restored to the exact same physical location. TTMmakes this rather easy but the caveat is the memory must be contiguous. Againfor simplity, we enforce that all kernel (pinned) BOs are contiguous andrestored to the same physical location.

Pinned external BOs in VRAM are restored on resume via the GPU.

Rebinds after suspend / resume¶

Most kernel BOs have GGTT mappings which must be restored during the resumeprocess. All user BOs are rebound after validation on their next use.

GGTT Internal API¶

structxe_ggtt_node¶

A node in GGTT.

struct xe_ggtt_node {    struct xe_ggtt *ggtt;    struct drm_mm_node base;    struct work_struct delayed_removal_work;    bool invalidate_on_remove;};

Description

Thisstructis allocated with xe_ggtt_insert_node(,_transform) or xe_ggtt_insert_bo(,_at).It will be deallocated usingxe_ggtt_node_remove().

structxe_ggtt_pt_ops¶

GGTT Page table operations Which can vary from platform to platform.

struct xe_ggtt_pt_ops {    u64 (*pte_encode_flags)(struct xe_bo *bo, u16 pat_index);    xe_ggtt_set_pte_fn ggtt_set_pte;    u64 (*ggtt_get_pte)(struct xe_ggtt *ggtt, u64 addr);};

structxe_ggtt¶

Main GGTT struct

struct xe_ggtt {    struct xe_tile *tile;    u64 start;    u64 size;#define XE_GGTT_FLAGS_64K BIT(0);    unsigned int flags;    struct xe_bo *scratch;    struct mutex lock;    u64 __iomem *gsm;    const struct xe_ggtt_pt_ops *pt_ops;    struct drm_mm mm;    unsigned int access_count;    struct workqueue_struct *wq;};

Description

In general, each tile can contains its own Global Graphics Translation Table(GGTT) instance.

u64xe_ggtt_start(structxe_ggtt*ggtt)¶

Get starting offset of GGTT.

u64xe_ggtt_size(structxe_ggtt*ggtt)¶

Get size of GGTT.

structxe_ggtt*xe_ggtt_alloc(structxe_tile*tile)¶

Allocate a GGTT for a givenxe_tile

intxe_ggtt_init_early(structxe_ggtt*ggtt)¶

Early GGTT initialization

voidxe_ggtt_node_remove(structxe_ggtt_node*node,boolinvalidate)¶

Remove axe_ggtt_node from the GGTT

intxe_ggtt_init(structxe_ggtt*ggtt)¶

Regular non-early GGTT initialization

voidxe_ggtt_shift_nodes(structxe_ggtt*ggtt,u64new_start)¶

Shift GGTT nodes to adjust for a change in usable address range.

structxe_ggtt_node*xe_ggtt_insert_node(structxe_ggtt*ggtt,u32size,u32align)¶

Insert axe_ggtt_node into the GGTT

size_txe_ggtt_node_pt_size(conststructxe_ggtt_node*node)¶

Get the size of page table entries needed to map a GGTT node.

voidxe_ggtt_map_bo(structxe_ggtt*ggtt,structxe_ggtt_node*node,structxe_bo*bo,u64pte)¶

Map the BO into GGTT

voidxe_ggtt_map_bo_unlocked(structxe_ggtt*ggtt,structxe_bo*bo)¶

Restore a mapping of a BO into GGTT

structxe_ggtt_node*xe_ggtt_insert_node_transform(structxe_ggtt*ggtt,structxe_bo*bo,u64pte_flags,u64size,u32align,xe_ggtt_transform_cbtransform,void*arg)¶

Insert a newly allocatedxe_ggtt_node into the GGTT

intxe_ggtt_insert_bo_at(structxe_ggtt*ggtt,structxe_bo*bo,u64start,u64end,structdrm_exec*exec)¶

Insert BO at a specific GGTT space

intxe_ggtt_insert_bo(structxe_ggtt*ggtt,structxe_bo*bo,structdrm_exec*exec)¶

Insert BO into GGTT

voidxe_ggtt_remove_bo(structxe_ggtt*ggtt,structxe_bo*bo)¶

Remove a BO from the GGTT

u64xe_ggtt_largest_hole(structxe_ggtt*ggtt,u64alignment,u64*spare)¶

Largest GGTT hole

voidxe_ggtt_assign(conststructxe_ggtt_node*node,u16vfid)¶

assign a GGTT region to the VF

intxe_ggtt_node_save(structxe_ggtt_node*node,void*dst,size_tsize,u16vfid)¶

Save axe_ggtt_node to a buffer.

intxe_ggtt_node_load(structxe_ggtt_node*node,constvoid*src,size_tsize,u16vfid)¶

Load axe_ggtt_node from a buffer.

intxe_ggtt_dump(structxe_ggtt*ggtt,structdrm_printer*p)¶

Dump GGTT for debug

u64xe_ggtt_print_holes(structxe_ggtt*ggtt,u64alignment,structdrm_printer*p)¶

Print holes

u64xe_ggtt_encode_pte_flags(structxe_ggtt*ggtt,structxe_bo*bo,u16pat_index)¶

Get PTE encoding flags for BO

u64xe_ggtt_read_pte(structxe_ggtt*ggtt,u64offset)¶

Read a PTE from the GGTT

u64xe_ggtt_node_addr(conststructxe_ggtt_node*node)¶

Getnode offset in GGTT.

u64xe_ggtt_node_size(conststructxe_ggtt_node*node)¶

Getnode allocation size.