Hardware workarounds¶

Hardware workarounds are register programming documented to be executed inthe driver that fall outside of the normal programming sequences for aplatform. There are some basic categories of workarounds, depending onhow/when they are applied:

• LRC workarounds: workarounds that touch registers that aresaved/restored to/from the HW context image. The list is emitted (via LoadRegister Immediate commands) once when initializing the device and saved inthe default context. That default context is then used on every contextcreation to have a “primed golden context”, i.e. a context image thatalready contains the changes needed to all the registers. Seedrivers/gpu/drm/xe/xe_lrc.c for default context handling.

• Engine workarounds: the list of these WAs is applied whenever the specificengine is reset. It’s also possible that a set of engine classes share acommon power domain and they are reset together. This happens on someplatforms with render and compute engines. In this case (at least) one ofthem need to keeep the workaround programming: the approach taken in thedriver is to tie those workarounds to the first compute/render engine thatis registered. When executing with GuC submission, engine resets areoutside of kernel driver control, hence the list of registers involved iswritten once, on engine initialization, and then passed to GuC, thatsaves/restores their values before/after the reset takes place. Seedrivers/gpu/drm/xe/xe_guc_ads.c for reference.

• GT workarounds: the list of these WAs is applied whenever these registersrevert to their default values: on GPU reset, suspend/resume[1], etc.

• Register whitelist: some workarounds need to be implemented in userspace,but need to touch privileged registers. The whitelist in the kernelinstructs the hardware to allow the access to happen. From the kernel side,this is just a special case of a MMIO workaround (as we write the list ofthese to/be-whitelisted registers to some special HW registers).

• Workaround batchbuffers: buffers that get executed automatically by thehardware on every HW context restore. These buffers are created andprogrammed in the default context so the hardware always go through thoseprogramming sequences when switching contexts. The support for workaroundbatchbuffers is enabled via these hardware mechanisms:

  1. INDIRECT_CTX (also known asmid context restore bb): A batchbufferand an offset are provided in the default context, pointing the hardwareto jump to that location when that offset is reached in the contextrestore. When a context is being restored, this is executed after thering context, in the middle (or beginning) of the engine context image.

  2. BB_PER_CTX_PTR (also known aspost context restore bb): Abatchbuffer is provided in the default context, pointing the hardware toa buffer to continue executing after the engine registers are restoredin a context restore sequence.

  Below is the timeline for a context restore sequence:

                    INDIRECT_CTX_OFFSET             |----------->|.------------.------------.-------------.------------.--------------.-----------.|Ring        | Engine     | Mid-context | Engine     | Post-context | Ring      ||Restore     | Restore (1)| BB Restore  | Restore (2)| BB Restore   | Execution |`------------'------------'-------------'------------'--------------'-----------'

• Other/OOB: There are WAs that, due to their nature, cannot be applied froma central place. Those are peppered around the rest of the code, as needed.There’s a central place to control which workarounds are enabled:drivers/gpu/drm/xe/xe_wa_oob.rules for GT workarounds anddrivers/gpu/drm/xe/xe_device_wa_oob.rules for device/SoC workarounds.These files only record which workarounds are enabled: during early deviceinitialization those rules are evaluated and recorded by the driver. Thenlater the driver checks withXE_GT_WA() andXE_DEVICE_WA() toimplement them.

Technically, some registers are powercontext saved & restored, so theysurvive a suspend/resume. In practice, writing them again is not toocostly and simplifies things, so it’s the approach taken in the driver.