18.1.Introduction¶

This document describes techniques that are useful for debugging issues withAMD Zen systems. It is intended for use by developers and technical usersto help identify and resolve issues.

18.2.S3 vs s2idle¶

On AMD systems, it’s not possible to simultaneously support suspend-to-RAM (S3)and suspend-to-idle (s2idle). To confirm which mode your system supports youcan look atcat/sys/power/mem_sleep. If it showss2idle[deep] thenS3 is supported. If it shows[s2idle] thens2idle issupported.

On systems that supportS3, the firmware will be utilized to put all hardware intothe appropriate low power state.

On systems that supports2idle, the kernel will be responsible for transitioning devicesinto the appropriate low power state. When all devices are in the appropriate lowpower state, the hardware will transition into a hardware sleep state.

After a suspend cycle you can tell how much time was spent in a hardware sleepstate by looking atcat/sys/power/suspend_stats/last_hw_sleep.

This flowchart explains how the AMD s2idle suspend flow works.

This flowchart explains how the amd s2idle resume flow works.

18.3.s2idle debugging tool¶

As there are a lot of places that problems can occur, a debugging tool has beencreated atamd-debug-toolsthat can help test for common problems and offer suggestions.

If you have an s2idle issue, it’s best to start with this and follow instructionsfrom its findings. If you continue to have an issue, raise a bug with thereport generated from this script todrm/amd gitlab.

18.4.Spurious s2idle wakeups from an IRQ¶

Spurious wakeups will generally have an IRQ set to/sys/power/pm_wakeup_irq.This can be matched to/proc/interrupts to determine what device woke the system.

If this isn’t enough to debug the problem, then the following sysfs filescan be set to add more verbosity to the wakeup process:

# echo 1 | sudo tee /sys/power/pm_debug_messages# echo 1 | sudo tee /sys/power/pm_print_times

After making those changes, the kernel will display messages that canbe traced back to kernel s2idle loop code as well as display any activeGPIO sources while waking up.

If the wakeup is caused by the ACPI SCI, additional ACPI debugging may beneeded. These commands can enable additional trace data:

# echo enable | sudo tee /sys/module/acpi/parameters/trace_state# echo 1 | sudo tee /sys/module/acpi/parameters/aml_debug_output# echo 0x0800000f | sudo tee /sys/module/acpi/parameters/debug_level# echo 0xffff0000 | sudo tee /sys/module/acpi/parameters/debug_layer

18.5.Spurious s2idle wakeups from a GPIO¶

If a GPIO is active when waking up the system ideally you would look at theschematic to determine what device it is associated with. If the schematicis not available, another tactic is to look at the ACPI_EVT() entryto determine what device is notified when that GPIO is active.

For a hypothetical example, say that GPIO 59 woke up the system. You canlook at the SSDT to determine what device is notified when GPIO 59 is active.

First convert the GPIO number into hex.

$ python3 -c "print(hex(59))"0x3b

Next determine which ACPI table has the_EVT entry. For example:

$ sudo grep EVT /sys/firmware/acpi/tables/SSDT*grep: /sys/firmware/acpi/tables/SSDT27: binary file matches

Decode this table:

$ sudo cp /sys/firmware/acpi/tables/SSDT27 .$ sudo iasl -d SSDT27

Then look at the table and find the matching entry for GPIO 0x3b.

Case (0x3B){    M000 (0x393B)    M460 ("    Notify (\\_SB.PCI0.GP17.XHC1, 0x02)\n", Zero, Zero, Zero, Zero, Zero, Zero)    Notify (\_SB.PCI0.GP17.XHC1, 0x02) // Device Wake}

You can see in this case that the device\_SB.PCI0.GP17.XHC1 is notifiedwhen GPIO 59 is active. It’s obvious this is an XHCI controller, but to go astep further you can figure out which XHCI controller it is by matching it toACPI.:

$ grep "PCI0.GP17.XHC1" /sys/bus/acpi/devices/*/path/sys/bus/acpi/devices/device:2d/path:\_SB_.PCI0.GP17.XHC1/sys/bus/acpi/devices/device:2e/path:\_SB_.PCI0.GP17.XHC1.RHUB/sys/bus/acpi/devices/device:2f/path:\_SB_.PCI0.GP17.XHC1.RHUB.PRT1/sys/bus/acpi/devices/device:30/path:\_SB_.PCI0.GP17.XHC1.RHUB.PRT1.CAM0/sys/bus/acpi/devices/device:31/path:\_SB_.PCI0.GP17.XHC1.RHUB.PRT1.CAM1/sys/bus/acpi/devices/device:32/path:\_SB_.PCI0.GP17.XHC1.RHUB.PRT2/sys/bus/acpi/devices/LNXPOWER:0d/path:\_SB_.PCI0.GP17.XHC1.PWRS

Here you can see it matches todevice:2d. Look at thephysical_nodeto determine what PCI device that actually is.

$ ls -l /sys/bus/acpi/devices/device:2d/physical_nodelrwxrwxrwx 1 root root 0 Feb 12 13:22 /sys/bus/acpi/devices/device:2d/physical_node -> ../../../../../pci0000:00/0000:00:08.1/0000:c2:00.4

So there you have it: the PCI device associated with this GPIO wakeup was0000:c2:00.4.

Theamd_s2idle.py script will capture most of these artifacts for you.

18.6.s2idle PM debug messages¶

During the s2idle flow on AMD systems, the ACPI LPS0 driver is responsibleto check all uPEP constraints. Failing uPEP constraints does not prevents0i3 entry. This means that if some constraints are not met, it is possiblethe kernel may attempt to enter s2idle even if there are some known issues.

To activate PM debugging, either specifypm_debug_messagess kernelcommand-line option at boot or write to/sys/power/pm_debug_messages.Unmet constraints will be displayed in the kernel log and can beviewed by logging tools that process kernel ring buffer likedmesg orjournalctl.”

If the system freezes on entry/exit before these messages are flushed, auseful debugging tactic is to unbind theamd_pmc driver to preventnotification to the platform to start s0i3 entry. This will stop thesystem from freezing on entry or exit and let you view all the failedconstraints.

cd /sys/bus/platform/drivers/amd_pmcls | grep AMD | sudo tee unbind

After doing this, run the suspend cycle and look specifically for errors around:

ACPI: LPI: Constraint not met; min power state:%s current power state:%s

18.7.Historical examples of s2idle issues¶

To help understand the types of issues that can occur and how to debug them,here are some historical examples of s2idle issues that have been resolved.

PM: dpm_run_callback(): acpi_subsys_suspend_noirq+0x0/0x50 returns -110amd_pmc AMDI0005:00: PM: failed to suspend noirq: error -110

18.8.Runtime power consumption issues¶

Runtime power consumption is influenced by many factors, including but notlimited to the configuration of the PCIe Active State Power Management (ASPM),the display brightness, the EPP policy of the CPU, and the power managementof the devices.

18.9.BIOS debug messages¶

Most OEM machines don’t have a serial UART for outputting kernel or BIOSdebug messages. However BIOS debug messages are useful for understandingboth BIOS bugs and bugs with the Linux kernel drivers that call BIOS AML.

As the BIOS on most OEM AMD systems are based off an AMD reference BIOS,the infrastructure used for exporting debugging messages is often the sameas AMD reference BIOS.

Method (M460, 7, Serialized)

M460 ("  OEM-ASL-PCIe Address (0x%X)._REG (%d %d)  PCSA = %d\n", DADR, Arg0, Arg1, PCSA, Zero, Zero)

extrace-0174 ex_trace_args         :  "  OEM-ASL-PCIe Address (0x%X)._REG (%d %d)  PCSA = %d\n", ec106000, 2, 1, 1, 0, 0

18.10.Random reboot issues¶

When a random reboot occurs, the high-level reason for the reboot is storedin a register that will persist onto the next boot.

There are 6 classes of reasons for the reboot:

• Software induced

• Power state transition

• Pin induced

• Hardware induced

• Remote reset

• Internal CPU event

This information is read by the kernel at bootup and printed intothe syslog. When a random reboot occurs this message can be helpfulto determine the next component to debug.