drm/nouveau NVIDIA GPU Driver

The drm/nouveau driver provides support for a wide range of NVIDIA GPUs,covering GeForce, Quadro, and Tesla series, from the NV04 architecture upto the latest Turing, Ampere, Ada families.

NVKM: NVIDIA Kernel Manager

The NVKM component serves as the core abstraction layer within the nouveaudriver, responsible for managing NVIDIA GPU hardware at the kernel level.NVKM provides a unified interface for handling various GPU architectures.

It enables resource management, power control, memory handling, and commandsubmission required for the proper functioning of NVIDIA GPUs under thenouveau driver.

NVKM plays a critical role in abstracting hardware complexities andproviding a consistent API to upper layers of the driver stack.

GSP Support

https://github.com/NVIDIA/open-gpu-kernel-modules/blob/535/src/nvidia/inc/kernel/gpu/gsp/message_queue_priv.h

The GSP command queue and status queue are message queues for thecommunication between software and GSP. The software submits the GSPRPC via the GSP command queue, GSP writes the status of the submittedRPC in the status queue.

A GSP message queue element consists of three parts:

  • message element header (structr535_gsp_msg), which mostly maintainsthe metadata for queuing the element.

  • RPC message header (structnvfw_gsp_rpc), which maintains the infoof the RPC. E.g., the RPC function number.

  • The payload, where the RPC message stays. E.g. the params of aspecific RPC function. Some RPC functions also have their headersin the payload. E.g. rm_alloc, rm_control.

The memory layout of a GSP message element can be illustrated below:

+------------------------+| Message Element Header ||    (r535_gsp_msg)      ||                        || (r535_gsp_msg.data)    ||          |             ||----------V-------------||    GSP RPC Header      ||    (nvfw_gsp_rpc)      ||                        || (nvfw_gsp_rpc.data)    ||          |             ||----------V-------------||       Payload          ||                        ||   header(optional)     ||        params          |+------------------------+

The max size of a message queue element is 16 pages (including theheaders). When a GSP message to be sent is larger than 16 pages, themessage should be split into multiple elements and sent accordingly.

In the bunch of the split elements, the first element has the expectedfunction number, while the rest of the elements are sent with thefunction number NV_VGPU_MSG_FUNCTION_CONTINUATION_RECORD.

GSP consumes the elements from the cmdq and always writes the resultback to the msgq. The result is also formed as split elements.

Terminology:

  • gsp_msg(msg): GSP message element (element header + GSP RPC header +payload)

  • gsp_rpc(rpc): GSP RPC (RPC header + payload)

  • gsp_rpc_buf: buffer for (GSP RPC header + payload)

  • gsp_rpc_len: size of (GSP RPC header + payload)

  • params_size: size of params in the payload

  • payload_size: size of (header if exists + params) in the payload

When sending a GSP RPC command, there can be multiple cases of handlingthe GSP RPC messages, which are the reply of GSP RPC commands, accordingto the requirement of the callers and the nature of the GSP RPC commands.

NVKM_GSP_RPC_REPLY_NOWAIT - If specified, immediately return to thecaller after the GSP RPC command is issued.

NVKM_GSP_RPC_REPLY_NOSEQ - If specified, exactly like NOWAITbut don’t emit RPC sequence number.

NVKM_GSP_RPC_REPLY_RECV - If specified, wait and receive the entire GSPRPC message after the GSP RPC command is issued.

NVKM_GSP_RPC_REPLY_POLL - If specified, wait for the specific reply anddiscard the reply before returning to the caller.