VGA Switcheroo

vga_switcheroo is the Linux subsystem for laptop hybrid graphics.These come in two flavors:

  • muxed: Dual GPUs with a multiplexer chip to switch outputs between GPUs.

  • muxless: Dual GPUs but only one of them is connected to outputs.The other one is merely used to offload rendering, its resultsare copied over PCIe into the framebuffer. On Linux this issupported with DRI PRIME.

Hybrid graphics started to appear in the late Naughties and were initiallyall muxed. Newer laptops moved to a muxless architecture for cost reasons.A notable exception is the MacBook Pro which continues to use a mux.Muxes come with varying capabilities: Some switch only the panel, otherscan also switch external displays. Some switch all display pins at oncewhile others can switch just the DDC lines. (To allow EDID probingfor the inactive GPU.) Also, muxes are often used to cut power to thediscrete GPU while it is not used.

DRM drivers register GPUs with vga_switcheroo, these are henceforth calledclients. The mux is called the handler. Muxless machines also register ahandler to control the power state of the discrete GPU, its ->switchtocallback is a no-op for obvious reasons. The discrete GPU is often equippedwith an HDA controller for the HDMI/DP audio signal, this will alsoregister as a client so that vga_switcheroo can take care of the correctsuspend/resume order when changing the discrete GPU’s power state. In totalthere can thus be up to three clients: Two vga clients (GPUs) and one audioclient (on the discrete GPU). The code is mostly prepared to supportmachines with more than two GPUs should they become available.

The GPU to which the outputs are currently switched is called theactive client in vga_switcheroo parlance. The GPU not in use is theinactive client. When the inactive client’s DRM driver is loaded,it will be unable to probe the panel’s EDID and hence depends onVBIOS to provide its display modes. If the VBIOS modes are bogus orif there is no VBIOS at all (which is common on the MacBook Pro),a client may alternatively request that the DDC lines are temporarilyswitched to it, provided that the handler supports this. Switchingonly the DDC lines and not the entire output avoids unnecessaryflickering.

Modes of Use

Manual switching and manual power control

In this mode of use, the file /sys/kernel/debug/vgaswitcheroo/switchcan be read to retrieve the current vga_switcheroo state and commandscan be written to it to change the state. The file appears as soon astwo GPU drivers and one handler have registered with vga_switcheroo.The following commands are understood:

  • OFF: Power off the device not in use.

  • ON: Power on the device not in use.

  • IGD: Switch to the integrated graphics device.Power on the integrated GPU if necessary, power off the discrete GPU.Prerequisite is that no user space processes (e.g. Xorg, alsactl)have opened device files of the GPUs or the audio client. If theswitch fails, the user may invoke lsof(8) or fuser(1) on /dev/dri/and /dev/snd/controlC1 to identify processes blocking the switch.

  • DIS: Switch to the discrete graphics device.

  • DIGD: Delayed switch to the integrated graphics device.This will perform the switch once the last user space process hasclosed the device files of the GPUs and the audio client.

  • DDIS: Delayed switch to the discrete graphics device.

  • MIGD: Mux-only switch to the integrated graphics device.Does not remap console or change the power state of either gpu.If the integrated GPU is currently off, the screen will turn black.If it is on, the screen will show whatever happens to be in VRAM.Either way, the user has to blindly enter the command to switch back.

  • MDIS: Mux-only switch to the discrete graphics device.

For GPUs whose power state is controlled by the driver’s runtime pm,the ON and OFF commands are a no-op (see next section).

For muxless machines, the IGD/DIS, DIGD/DDIS and MIGD/MDIS commandsshould not be used.

Driver power control

In this mode of use, the discrete GPU automatically powers up and down atthe discretion of the driver’s runtime pm. On muxed machines, the user maystill influence the muxer state by way of the debugfs interface, howeverthe ON and OFF commands become a no-op for the discrete GPU.

This mode is the default on Nvidia HybridPower/Optimus and ATI PowerXpress.Specifying nouveau.runpm=0, radeon.runpm=0 or amdgpu.runpm=0 on the kernelcommand line disables it.

After the GPU has been suspended, the handler needs to be called to cutpower to the GPU. Likewise it needs to reinstate power before the GPUcan resume. This is achieved byvga_switcheroo_init_domain_pm_ops(),which augments the GPU’s suspend/resume functions by the requisitecalls to the handler.

When the audio device resumes, the GPU needs to be woken. This is achievedby a PCI quirk which callsdevice_link_add() to declare a dependency on theGPU. That way, the GPU is kept awake whenever and as long as the audiodevice is in use.

On muxed machines, if the mux is initially switched to the discrete GPU,the user ends up with a black screen when the GPU powers down after boot.As a workaround, the mux is forced to the integrated GPU on runtime suspend,cf.https://bugs.freedesktop.org/show_bug.cgi?id=75917

API

Public functions

intvga_switcheroo_register_handler(conststructvga_switcheroo_handler*handler,enumvga_switcheroo_handler_flags_thandler_flags)

register handler

Parameters

conststructvga_switcheroo_handler*handler

handler callbacks

enumvga_switcheroo_handler_flags_thandler_flags

handler flags

Description

Register handler. Enable vga_switcheroo if two vga clients have alreadyregistered.

Return

0 on success, -EINVAL if a handler was already registered.

voidvga_switcheroo_unregister_handler(void)

unregister handler

Parameters

void

no arguments

Description

Unregister handler. Disable vga_switcheroo.

enumvga_switcheroo_handler_flags_tvga_switcheroo_handler_flags(void)

obtain handler flags

Parameters

void

no arguments

Description

Helper for clients to obtain the handler flags bitmask.

Return

Handler flags. A value of 0 means that no handler is registeredor that the handler has no special capabilities.

intvga_switcheroo_register_client(structpci_dev*pdev,conststructvga_switcheroo_client_ops*ops,booldriver_power_control)

register vga client

Parameters

structpci_dev*pdev

client pci device

conststructvga_switcheroo_client_ops*ops

client callbacks

booldriver_power_control

whether power state is controlled by the driver’sruntime pm

Description

Register vga client (GPU). Enable vga_switcheroo if another GPU and ahandler have already registered. The power state of the client is assumedto be ON. Beforehand,vga_switcheroo_client_probe_defer() shall be calledto ensure that all prerequisites are met.

Return

0 on success, -ENOMEM on memory allocation error.

intvga_switcheroo_register_audio_client(structpci_dev*pdev,conststructvga_switcheroo_client_ops*ops,structpci_dev*vga_dev)

register audio client

Parameters

structpci_dev*pdev

client pci device

conststructvga_switcheroo_client_ops*ops

client callbacks

structpci_dev*vga_dev

pci device which is bound to current audio client

Description

Register audio client (audio device on a GPU). The client is assumedto use runtime PM. Beforehand,vga_switcheroo_client_probe_defer()shall be called to ensure that all prerequisites are met.

Return

0 on success, -ENOMEM on memory allocation error, -EINVAL on gettingclient id error.

boolvga_switcheroo_client_probe_defer(structpci_dev*pdev)

whether to defer probing a given client

Parameters

structpci_dev*pdev

client pci device

Description

Determine whether any prerequisites are not fulfilled to probe a givenclient. Drivers shall invoke this early on in their ->probe callbackand return-EPROBE_DEFER if it evaluates totrue. Thou shalt notregister the client ere thou hast called this.

Return

true if probing should be deferred, otherwisefalse.

enumvga_switcheroo_statevga_switcheroo_get_client_state(structpci_dev*pdev)

obtain power state of a given client

Parameters

structpci_dev*pdev

client pci device

Description

Obtain power state of a given client as seen from vga_switcheroo.The function is only called from hda_intel.c.

Return

Power state.

voidvga_switcheroo_unregister_client(structpci_dev*pdev)

unregister client

Parameters

structpci_dev*pdev

client pci device

Description

Unregister client. Disable vga_switcheroo if this is a vga client (GPU).

voidvga_switcheroo_client_fb_set(structpci_dev*pdev,structfb_info*info)

set framebuffer of a given client

Parameters

structpci_dev*pdev

client pci device

structfb_info*info

framebuffer

Description

Set framebuffer of a given client. The console will be remapped to thison switching.

intvga_switcheroo_lock_ddc(structpci_dev*pdev)

temporarily switch DDC lines to a given client

Parameters

structpci_dev*pdev

client pci device

Description

Temporarily switch DDC lines to the client identified bypdev(but leave the outputs otherwise switched to where they are).This allows the inactive client to probe EDID. The DDC lines mustafterwards be switched back by callingvga_switcheroo_unlock_ddc(),even if this function returns an error.

Return

Previous DDC owner on success or a negative int on error.Specifically,-ENODEV if no handler has registered or if the handlerdoes not support switching the DDC lines. Also, a negative valuereturned by the handler is propagated back to the caller.The return value has merely an informational purpose for any callerwhich might be interested in it. It is acceptable to ignore the returnvalue and simply rely on the result of the subsequent EDID probe,which will beNULL if DDC switching failed.

intvga_switcheroo_unlock_ddc(structpci_dev*pdev)

switch DDC lines back to previous owner

Parameters

structpci_dev*pdev

client pci device

Description

Switch DDC lines back to the previous owner after callingvga_switcheroo_lock_ddc(). This must be called even ifvga_switcheroo_lock_ddc() returned an error.

Return

Previous DDC owner on success (i.e. the client identifier ofpdev)or a negative int on error.Specifically,-ENODEV if no handler has registered or if the handlerdoes not support switching the DDC lines. Also, a negative valuereturned by the handler is propagated back to the caller.Finally, invoking this function without callingvga_switcheroo_lock_ddc()first is not allowed and will result in-EINVAL.

intvga_switcheroo_process_delayed_switch(void)

helper for delayed switching

Parameters

void

no arguments

Description

Process a delayed switch if one is pending.

Return

0 on success. -EINVAL if no delayed switch is pending, if the clienthas unregistered in the meantime or if there are other clients blocking theswitch. If the actual switch fails, an error is reported and 0 is returned.

intvga_switcheroo_init_domain_pm_ops(structdevice*dev,structdev_pm_domain*domain)

helper for driver power control

Parameters

structdevice*dev

vga client device

structdev_pm_domain*domain

power domain

Description

Helper for GPUs whose power state is controlled by the driver’s runtime pm.After the GPU has been suspended, the handler needs to be called to cutpower to the GPU. Likewise it needs to reinstate power before the GPUcan resume. To this end, this helper augments the suspend/resume functionsby the requisite calls to the handler. It needs only be called on platformswhere the power switch is separate to the device being powered down.

Public structures

structvga_switcheroo_handler

handler callbacks

Definition:

struct vga_switcheroo_handler {    int (*init)(void);    int (*switchto)(enum vga_switcheroo_client_id id);    int (*switch_ddc)(enum vga_switcheroo_client_id id);    int (*power_state)(enum vga_switcheroo_client_id id, enum vga_switcheroo_state state);    enum vga_switcheroo_client_id (*get_client_id)(struct pci_dev *pdev);};

Members

init

initialize handler.Optional. This gets called when vga_switcheroo is enabled, i.e. whentwo vga clients have registered. It allows the handler to performsome delayed initialization that depends on the existence of thevga clients. Currently only the radeon and amdgpu drivers use this.The return value is ignored

switchto

switch outputs to given client.Mandatory. For muxless machines this should be a no-op. Returning 0denotes success, anything else failure (in which case the switch isaborted)

switch_ddc

switch DDC lines to given client.Optional. Should return the previous DDC owner on success or anegative int on failure

power_state

cut or reinstate power of given client.Optional. The return value is ignored

get_client_id

determine if given pci device is integrated or discrete GPU.Mandatory

Description

Handler callbacks. The multiplexer itself. Theswitchto andget_client_idmethods are mandatory, all others may be set to NULL.

structvga_switcheroo_client_ops

client callbacks

Definition:

struct vga_switcheroo_client_ops {    void (*set_gpu_state)(struct pci_dev *dev, enum vga_switcheroo_state);    void (*reprobe)(struct pci_dev *dev);    bool (*can_switch)(struct pci_dev *dev);    void (*gpu_bound)(struct pci_dev *dev, enum vga_switcheroo_client_id);};

Members

set_gpu_state

do the equivalent of suspend/resume for the card.Mandatory. This should not cut power to the discrete GPU,which is the job of the handler

reprobe

poll outputs.Optional. This gets called after waking the GPU and switchingthe outputs to it

can_switch

check if the device is in a position to switch now.Mandatory. The client should return false if a user space processhas one of its device files open

gpu_bound

notify the client id to audio client when the GPU is bound.

Description

Client callbacks. A client can be either a GPU or an audio device on a GPU.Theset_gpu_state andcan_switch methods are mandatory,reprobe may beset to NULL. For audio clients, thereprobe member is bogus.OTOH,gpu_bound is only for audio clients, and not used for GPU clients.

Public constants

enumvga_switcheroo_handler_flags_t

handler flags bitmask

Constants

VGA_SWITCHEROO_CAN_SWITCH_DDC

whether the handler is able to switch theDDC lines separately. This signals to clients that they should calldrm_get_edid_switcheroo() to probe the EDID

VGA_SWITCHEROO_NEEDS_EDP_CONFIG

whether the handler is unable to switchthe AUX channel separately. This signals to clients that the activeGPU needs to train the link and communicate the link parameters to theinactive GPU (mediated by vga_switcheroo). The inactive GPU may thenskip the AUX handshake and set up its output with these pre-calibratedvalues (DisplayPort specification v1.1a, section 2.5.3.3)

Description

Handler flags bitmask. Used by handlers to declare their capabilities uponregistering with vga_switcheroo.

enumvga_switcheroo_client_id

client identifier

Constants

VGA_SWITCHEROO_UNKNOWN_ID

initial identifier assigned to vga clients.Determining the id requires the handler, so GPUs are given theirtrue id in a delayed fashion invga_switcheroo_enable()

VGA_SWITCHEROO_IGD

integrated graphics device

VGA_SWITCHEROO_DIS

discrete graphics device

VGA_SWITCHEROO_MAX_CLIENTS

currently no more than two GPUs are supported

Description

Client identifier. Audio clients use the same identifier & 0x100.

enumvga_switcheroo_state

client power state

Constants

VGA_SWITCHEROO_OFF

off

VGA_SWITCHEROO_ON

on

VGA_SWITCHEROO_NOT_FOUND

client has not registered with vga_switcheroo.Only used invga_switcheroo_get_client_state() which in turn is onlycalled from hda_intel.c

Description

Client power state.

Private structures

structvgasr_priv

vga_switcheroo private data

Definition:

struct vgasr_priv {    bool active;    bool delayed_switch_active;    enum vga_switcheroo_client_id delayed_client_id;    struct dentry *debugfs_root;    int registered_clients;    struct list_head clients;    const struct vga_switcheroo_handler *handler;    enum vga_switcheroo_handler_flags_t handler_flags;    struct mutex mux_hw_lock;    int old_ddc_owner;};

Members

active

whether vga_switcheroo is enabled.Prerequisite is the registration of two GPUs and a handler

delayed_switch_active

whether a delayed switch is pending

delayed_client_id

client to which a delayed switch is pending

debugfs_root

directory for vga_switcheroo debugfs interface

registered_clients

number of registered GPUs(counting only vga clients, not audio clients)

clients

list of registered clients

handler

registered handler

handler_flags

flags of registered handler

mux_hw_lock

protects mux state(in particular while DDC lines are temporarily switched)

old_ddc_owner

client to which DDC lines will be switched back on unlock

Description

vga_switcheroo private data. Currently only one vga_switcheroo instanceper system is supported.

structvga_switcheroo_client

registered client

Definition:

struct vga_switcheroo_client {    struct pci_dev *pdev;    struct fb_info *fb_info;    enum vga_switcheroo_state pwr_state;    const struct vga_switcheroo_client_ops *ops;    enum vga_switcheroo_client_id id;    bool active;    bool driver_power_control;    struct list_head list;    struct pci_dev *vga_dev;};

Members

pdev

client pci device

fb_info

framebuffer to which console is remapped on switching

pwr_state

current power state if manual power control is used.For driver power control, callvga_switcheroo_pwr_state().

ops

client callbacks

id

client identifier. Determining the id requires the handler,so gpus are initially assigned VGA_SWITCHEROO_UNKNOWN_IDand later given their true id invga_switcheroo_enable()

active

whether the outputs are currently switched to this client

driver_power_control

whether power state is controlled by the driver’sruntime pm. If true, writing ON and OFF to the vga_switcheroo debugfsinterface is a no-op so as not to interfere with runtime pm

list

client list

vga_dev

pci device, indicate which GPU is bound to current audio client

Description

Registered client. A client can be either a GPU or an audio device on a GPU.For audio clients, thefb_info andactive members are bogus. For GPUclients, thevga_dev is bogus.

Handlers

apple-gmux Handler

gmux is a microcontroller built into the MacBook Pro to support dual GPUs:ALattice XP2 on pre-retinas, aRenesas R4F2113 on pre-T2 retinas.

On T2 Macbooks, the gmux is part of the T2 Coprocessor’s SMC. The SMC hasan I2C connection to aNXP PCAL6524 GPIO expander, which enables/disablesthe voltage regulators of the discrete GPU, drives the display panel power,and has a GPIO to switch the eDP mux. The Intel CPU can interact withgmux through MMIO, similar to how the main SMC interface is controlled.

(The MacPro6,1 2013 also has a gmux, however it is unclear why since it hasdual GPUs but no built-in display.)

gmux is connected to the LPC bus of the southbridge. Its I/O ports areaccessed differently depending on the microcontroller: Driver functionsto access a pre-retina gmux are infixed_pio_, those for a pre-T2retina gmux are infixed_index_, and those on T2 Macs are infixedwith_mmio_.

gmux is also connected to a GPIO pin of the southbridge and thereby is ableto trigger an ACPI GPE. ACPI name GMGP holds this GPIO pin’s number. On theMBP5 2008/09 it’s GPIO pin 22 of the Nvidia MCP79, on following generationsit’s GPIO pin 6 of the Intel PCH, on MMIO gmux’s it’s pin 21.

The GPE merely signals that an interrupt occurred, the actual type of eventis identified by reading a gmux register.

In addition to the GMGP name, gmux’s ACPI device also has two methods GMSPand GMLV. GMLV likely means “GMUX Level”, and reads the value of the GPIO,while GMSP likely means “GMUX Set Polarity”, and seems to write to the GPIO’svalue. On newer Macbooks (This was introduced with or sometime before theMacBookPro14,3), the ACPI GPE method differentiates between the OS type: OnDarwin, only a notification is signaled, whereas on other OSes, the GPIO’svalue is read and then inverted.

Because Linux masquerades as Darwin, it ends up in the notification-only codepath. On MMIO gmux’s, this seems to lead to us being unable to clear interrupts,unless we call GMSP(0). Without this, there is a flood of status=0 interruptsthat can’t be cleared. This issue seems to be unique to MMIO gmux’s.

Graphics mux

On pre-retinas, the LVDS outputs of both GPUs feed into gmux which muxeseither of them to the panel. One of the tricks gmux has up its sleeve isto lengthen the blanking interval of its output during a switch tosynchronize it with the GPU switched to. This allows for a flicker-freeswitch that is imperceptible by the user (US 8,687,007 B2).

On retinas, muxing is no longer done by gmux itself, but by a separatechip which is controlled by gmux. The chip is triple sourced, it iseither anNXP CBTL06142,TI HD3SS212 orPericom PI3VDP12412.The panel is driven with eDP instead of LVDS since the pixel clockrequired for retina resolution exceeds LVDS’ limits.

Pre-retinas are able to switch the panel’s DDC pins separately.This is handled by aTI SN74LV4066A which is controlled by gmux.The inactive GPU can thus probe the panel’s EDID without switching overthe entire panel. Retinas lack this functionality as the chips used foreDP muxing are incapable of switching the AUX channel separately (seethe linked data sheets, Pericom would be capable but this is unused).However the retina panel has the NO_AUX_HANDSHAKE_LINK_TRAINING bit setin its DPCD, allowing the inactive GPU to skip the AUX handshake andset up the output with link parameters pre-calibrated by the active GPU.

The external DP port is only fully switchable on the first two unibodyMacBook Pro generations, MBP5 2008/09 and MBP6 2010. This is done by anNXP CBTL06141 which is controlled by gmux. It’s the predecessor of theeDP mux on retinas, the difference being support for 2.7 versus 5.4 Gbit/s.

The following MacBook Pro generations replaced the external DP port with acombined DP/Thunderbolt port and lost the ability to switch it between GPUs,connecting it either to the discrete GPU or the Thunderbolt controller.Oddly enough, while the full port is no longer switchable, AUX and HPDare still switchable by way of anNXP CBTL03062 (on pre-retinasMBP8 2011 and MBP9 2012) or twoTI TS3DS10224 (on pre-t2 retinas) underthe control of gmux. Since the integrated GPU is missing the main link,external displays appear to it as phantoms which fail to link-train.

gmux receives the HPD signal of all display connectors and sends aninterrupt on hotplug. On generations which cannot switch external ports,the discrete GPU can then be woken to drive the newly connected display.The ability to switch AUX on these generations could be used to improvereliability of hotplug detection by having the integrated GPU poll theports while the discrete GPU is asleep, but currently we do not make useof this feature.

Our switching policy for the external port is that on those generationswhich are able to switch it fully, the port is switched together with thepanel when IGD / DIS commands are issued to vga_switcheroo. It is thuspossible to drive e.g. a beamer on battery power with the integrated GPU.The user may manually switch to the discrete GPU if more performance isneeded.

On all newer generations, the external port can only be driven by thediscrete GPU. If a display is plugged in while the panel is switched tothe integrated GPU,both GPUs will be in use for maximum performance.To decrease power consumption, the user may manually switch to thediscrete GPU, thereby suspending the integrated GPU.

gmux’ initial switch state on bootup is user configurable via the EFIvariablegpu-power-prefs-fa4ce28d-b62f-4c99-9cc3-6815686e30f9 (5th byte,1 = IGD, 0 = DIS). Based on this setting, the EFI firmware tells gmux toswitch the panel and the external DP connector and allocates a framebufferfor the selected GPU.

Power control

gmux is able to cut power to the discrete GPU. It automatically takes careof the correct sequence to tear down and bring up the power rails forcore voltage, VRAM and PCIe.

Backlight control

On single GPU MacBooks, the PWM signal for the backlight is generated bythe GPU. On dual GPU MacBook Pros by contrast, either GPU may be suspendedto conserve energy. Hence the PWM signal needs to be generated by a separatebacklight driver which is controlled by gmux. The earliest generationMBP5 2008/09 uses aTI LP8543 backlight driver. Newer modelsuse aTI LP8545 or a TI LP8548.

Public functions

boolapple_gmux_detect(structpnp_dev*pnp_dev,enumapple_gmux_type*type_ret)

detect if gmux is built into the machine

Parameters

structpnp_dev*pnp_dev

Device to probe or NULL to use the first matching device

enumapple_gmux_type*type_ret

Returns (by reference) the apple_gmux_type of the device

Description

Detect if a supported gmux device is present by actually probing it.This avoids the false positives returned on some models byapple_gmux_present().

Return

true if a supported gmux ACPI device is detected and the kernelwas configured with CONFIG_APPLE_GMUX,false otherwise.

boolapple_gmux_present(void)

check if gmux ACPI device is present

Parameters

void

no arguments

Description

Drivers may use this to activate quirks specific to dual GPU MacBook Prosand Mac Pros, e.g. for deferred probing, runtime pm and backlight.

Return

true if gmux ACPI device is present and the kernel was configuredwith CONFIG_APPLE_GMUX,false otherwise.