Usage¶

The earliest point in time when device links can be added is afterdevice_add() has been called for the supplier anddevice_initialize() has been called for the consumer.

It is legal to add them later, but care must be taken that the systemremains in a consistent state: E.g. a device link cannot be added inthe midst of a suspend/resume transition, so either commencement ofsuch a transition needs to be prevented withlock_system_sleep(),or the device link needs to be added from a function which is guaranteednot to run in parallel to a suspend/resume transition, such as from adevice->probe callback or a boot-time PCI quirk.

Another example for an inconsistent state would be a device link thatrepresents a driver presence dependency, yet is added from the consumer’s->probe callback while the supplier hasn’t started to probe yet: Had thedriver core known about the device link earlier, it wouldn’t have probed theconsumer in the first place. The onus is thus on the consumer to checkpresence of the supplier after adding the link, and defer probing onnon-presence. [Note that it is valid to create a link from the consumer’s->probe callback while the supplier is still probing, but the consumer mustknow that the supplier is functional already at the link creation time (that isthe case, for instance, if the consumer has just acquired some resources thatwould not have been available had the supplier not been functional then).]

If a device link withDL_FLAG_STATELESS set (i.e. a stateless device link)is added in the->probe callback of the supplier or consumer driver, it istypically deleted in its->remove callback for symmetry. That way, if thedriver is compiled as a module, the device link is added on module load andorderly deleted on unload. The same restrictions that apply to device linkaddition (e.g. exclusion of a parallel suspend/resume transition) apply equallyto deletion. Device links managed by the driver core are deleted automaticallyby it.

Several flags may be specified on device link addition, two of whichhave already been mentioned above:DL_FLAG_STATELESS to express that nodriver presence dependency is needed (but only correct suspend/resume andshutdown ordering) andDL_FLAG_PM_RUNTIME to express that runtime PMintegration is desired.

Two other flags are specifically targeted at use cases where the devicelink is added from the consumer’s->probe callback:DL_FLAG_RPM_ACTIVEcan be specified to runtime resume the supplier and prevent it from suspendingbefore the consumer is runtime suspended.DL_FLAG_AUTOREMOVE_CONSUMERcauses the device link to be automatically purged when the consumer fails toprobe or later unbinds.

Similarly, when the device link is added from supplier’s->probe callback,DL_FLAG_AUTOREMOVE_SUPPLIER causes the device link to be automaticallypurged when the supplier fails to probe or later unbinds.

If neitherDL_FLAG_AUTOREMOVE_CONSUMER norDL_FLAG_AUTOREMOVE_SUPPLIERis set,DL_FLAG_AUTOPROBE_CONSUMER can be used to request the driver coreto probe for a driver for the consumer driver on the link automatically aftera driver has been bound to the supplier device.

Note, however, that any combinations ofDL_FLAG_AUTOREMOVE_CONSUMER,DL_FLAG_AUTOREMOVE_SUPPLIER orDL_FLAG_AUTOPROBE_CONSUMER withDL_FLAG_STATELESS are invalid and cannot be used.

Limitations¶

Driver authors should be aware that a driver presence dependency for manageddevice links (i.e. whenDL_FLAG_STATELESS is not specified on link addition)may cause probing of the consumer to be deferred indefinitely. This can becomea problem if the consumer is required to probe before a certain initcall levelis reached. Worse, if the supplier driver is blacklisted or missing, theconsumer will never be probed.

Moreover, managed device links cannot be deleted directly. They are deletedby the driver core when they are not necessary any more in accordance with theDL_FLAG_AUTOREMOVE_CONSUMER andDL_FLAG_AUTOREMOVE_SUPPLIER flags.However, stateless device links (i.e. device links withDL_FLAG_STATELESSset) are expected to be removed by whoever calleddevice_link_add()to add them with the help of eitherdevice_link_del() ordevice_link_remove().

PassingDL_FLAG_RPM_ACTIVE along withDL_FLAG_STATELESS todevice_link_add() may cause the PM-runtime usage counter of thesupplier device to remain nonzero after a subsequent invocation of eitherdevice_link_del() ordevice_link_remove() to remove thedevice link returned by it. This happens ifdevice_link_add() iscalled twice in a row for the same consumer-supplier pair without removing thelink between these calls, in which case allowing the PM-runtime usage counterof the supplier to drop on an attempt to remove the link may cause it to besuspended while the consumer is still PM-runtime-active and that has to beavoided. [To work around this limitation it is sufficient to let the consumerruntime suspend at least once, or callpm_runtime_set_suspended() forit with PM-runtime disabled, between thedevice_link_add() anddevice_link_del() ordevice_link_remove() calls.]

Sometimes drivers depend on optional resources. They are able to operatein a degraded mode (reduced feature set or performance) when those resourcesare not present. An example is an SPI controller that can use a DMA engineor work in PIO mode. The controller can determine presence of the optionalresources at probe time but on non-presence there is no way to know whetherthey will become available in the near future (due to a supplier driverprobing) or never. Consequently it cannot be determined whether to deferprobing or not. It would be possible to notify drivers when optionalresources become available after probing, but it would come at a high costfor drivers as switching between modes of operation at runtime based on theavailability of such resources would be much more complex than a mechanismbased on probe deferral. In any case optional resources are beyond thescope of device links.

Examples¶

• An MMU device exists alongside a busmaster device, both are in the samepower domain. The MMU implements DMA address translation for the busmasterdevice and shall be runtime resumed and kept active whenever and as longas the busmaster device is active. The busmaster device’s driver shallnot bind before the MMU is bound. To achieve this, a device link withruntime PM integration is added from the busmaster device (consumer)to the MMU device (supplier). The effect with regards to runtime PMis the same as if the MMU was the parent of the master device.

  The fact that both devices share the same power domain would normallysuggest usage of astructdev_pm_domain orstructgeneric_pm_domain,however these are not independent devices that happen to share a powerswitch, but rather the MMU device serves the busmaster device and isuseless without it. A device link creates a synthetic hierarchicalrelationship between the devices and is thus more apt.

• A Thunderbolt host controller comprises a number of PCIe hotplug portsand an NHI device to manage the PCIe switch. On resume from system sleep,the NHI device needs to re-establish PCI tunnels to attached devicesbefore the hotplug ports can resume. If the hotplug ports were childrenof the NHI, this resume order would automatically be enforced by thePM core, but unfortunately they’re aunts. The solution is to adddevice links from the hotplug ports (consumers) to the NHI device(supplier). A driver presence dependency is not necessary for thisuse case.

• Discrete GPUs in hybrid graphics laptops often feature an HDA controllerfor HDMI/DP audio. In the device hierarchy the HDA controller is a siblingof the VGA device, yet both share the same power domain and the HDAcontroller is only ever needed when an HDMI/DP display is attached to theVGA device. A device link from the HDA controller (consumer) to theVGA device (supplier) aptly represents this relationship.

• ACPI allows definition of a device start order by way of _DEP objects.A classical example is when ACPI power management methods on one deviceare implemented in terms of I²C accesses and require a specificI²C controller to be present and functional for the powermanagement of the device in question to work.

• In some SoCs a functional dependency exists from display, video codec andvideo processing IP cores on transparent memory access IP cores that handleburst access and compression/decompression.

Alternatives¶

• Astructdev_pm_domain can be used to override the bus,class or device type callbacks. It is intended for devices sharinga single on/off switch, however it does not guarantee a specificsuspend/resume ordering, this needs to be implemented separately.It also does not by itself track the runtime PM status of the involveddevices and turn off the power switch only when all of them are runtimesuspended. Furthermore it cannot be used to enforce a specific shutdownordering or a driver presence dependency.
• Astructgeneric_pm_domain is a lot more heavyweight than adevice link and does not allow for shutdown ordering or driver presencedependencies. It also cannot be used on ACPI systems.

Implementation¶

The device hierarchy, which – as the name implies – is a tree,becomes a directed acyclic graph once device links are added.

Ordering of these devices during suspend/resume is determined by thedpm_list. During shutdown it is determined by the devices_kset. Withno device links present, the two lists are a flattened, one-dimensionalrepresentations of the device tree such that a device is placed behindall its ancestors. That is achieved by traversing the ACPI namespaceor OpenFirmware device tree top-down and appending devices to the listsas they are discovered.

Once device links are added, the lists need to satisfy the additionalconstraint that a device is placed behind all its suppliers, recursively.To ensure this, upon addition of the device link the consumer and theentire sub-graph below it (all children and consumers of the consumer)are moved to the end of the list. (Call todevice_reorder_to_tail()fromdevice_link_add().)

To prevent introduction of dependency loops into the graph, it isverified upon device link addition that the supplier is not dependenton the consumer or any children or consumers of the consumer.(Call todevice_is_dependent() fromdevice_link_add().)If that constraint is violated,device_link_add() will returnNULL and aWARNING will be logged.

Notably this also prevents the addition of a device link from a parentdevice to a child. However the converse is allowed, i.e. a device linkfrom a child to a parent. Since the driver core already guaranteescorrect suspend/resume and shutdown ordering between parent and child,such a device link only makes sense if a driver presence dependency isneeded on top of that. In this case driver authors should weighcarefully if a device link is at all the right tool for the purpose.A more suitable approach might be to simply use deferred probing oradd a device flag causing the parent driver to be probed before thechild one.

State machine¶

enumdevice_link_state¶

Device link states.

Constants

DL_STATE_NONE

The presence of the drivers is not being tracked.

DL_STATE_DORMANT

None of the supplier/consumer drivers is present.

DL_STATE_AVAILABLE

The supplier driver is present, but the consumer is not.

DL_STATE_CONSUMER_PROBE

The consumer is probing (supplier driver present).

DL_STATE_ACTIVE

Both the supplier and consumer drivers are present.

DL_STATE_SUPPLIER_UNBIND

The supplier driver is unbinding.

                .=============================.                |                             |                v                             |DORMANT <=> AVAILABLE <=> CONSUMER_PROBE => ACTIVE   ^                                          |   |                                          |   '============ SUPPLIER_UNBIND <============'

• The initial state of a device link is automatically determined bydevice_link_add() based on the driver presence on the supplierand consumer. If the link is created before any devices are probed, itis set toDL_STATE_DORMANT.
• When a supplier device is bound to a driver, links to its consumersprogress toDL_STATE_AVAILABLE.(Call todevice_links_driver_bound() fromdriver_bound().)
• Before a consumer device is probed, presence of supplier drivers isverified by checking the consumer device is not in the wait_for_supplierslist and by checking that links to suppliers are inDL_STATE_AVAILABLEstate. The state of the links is updated toDL_STATE_CONSUMER_PROBE.(Call todevice_links_check_suppliers() fromreally_probe().)This prevents the supplier from unbinding.(Call towait_for_device_probe() fromdevice_links_unbind_consumers().)
• If the probe fails, links to suppliers revert back toDL_STATE_AVAILABLE.(Call todevice_links_no_driver() fromreally_probe().)
• If the probe succeeds, links to suppliers progress toDL_STATE_ACTIVE.(Call todevice_links_driver_bound() fromdriver_bound().)
• When the consumer’s driver is later on removed, links to suppliers revertback toDL_STATE_AVAILABLE.(Call to__device_links_no_driver() fromdevice_links_driver_cleanup(), which in turn is called from__device_release_driver().)
• Before a supplier’s driver is removed, links to consumers that are notbound to a driver are updated toDL_STATE_SUPPLIER_UNBIND.(Call todevice_links_busy() from__device_release_driver().)This prevents the consumers from binding.(Call todevice_links_check_suppliers() fromreally_probe().)Consumers that are bound are freed from their driver; consumers that areprobing are waited for until they are done.(Call todevice_links_unbind_consumers() from__device_release_driver().)Once all links to consumers are inDL_STATE_SUPPLIER_UNBIND state,the supplier driver is released and the links revert toDL_STATE_DORMANT.(Call todevice_links_driver_cleanup() from__device_release_driver().)

API¶

structdevice_link *device_link_add(structdevice * consumer, structdevice * supplier, u32 flags)¶

Create a link between two devices.

Parameters

structdevice*consumer

Consumer end of the link.

structdevice*supplier

Supplier end of the link.

u32flags

Link flags.

Description

The caller is responsible for the proper synchronization of the link creationwith runtime PM. First, setting the DL_FLAG_PM_RUNTIME flag will cause theruntime PM framework to take the link into account. Second, if theDL_FLAG_RPM_ACTIVE flag is set in addition to it, the supplier devices willbe forced into the active metastate and reference-counted upon the creationof the link. If DL_FLAG_PM_RUNTIME is not set, DL_FLAG_RPM_ACTIVE will beignored.

If DL_FLAG_STATELESS is set inflags, the caller of this function isexpected to release the link returned by it directly with the help of eitherdevice_link_del() ordevice_link_remove().

If that flag is not set, however, the caller of this function is handing themanagement of the link over to the driver core entirely and its return valuecan only be used to check whether or not the link is present. In that case,the DL_FLAG_AUTOREMOVE_CONSUMER and DL_FLAG_AUTOREMOVE_SUPPLIER device linkflags can be used to indicate to the driver core when the link can be safelydeleted. Namely, setting one of them inflags indicates to the driver corethat the link is not going to be used (by the given caller of this function)after unbinding the consumer or supplier driver, respectively, from itsdevice, so the link can be deleted at that point. If none of them is set,the link will be maintained until one of the devices pointed to by it (eitherthe consumer or the supplier) is unregistered.

Also, if DL_FLAG_STATELESS, DL_FLAG_AUTOREMOVE_CONSUMER andDL_FLAG_AUTOREMOVE_SUPPLIER are not set inflags (that is, a persistentmanaged device link is being added), the DL_FLAG_AUTOPROBE_CONSUMER flag canbe used to request the driver core to automaticall probe for a consmerdriver after successfully binding a driver to the supplier device.

The combination of DL_FLAG_STATELESS and one of DL_FLAG_AUTOREMOVE_CONSUMER,DL_FLAG_AUTOREMOVE_SUPPLIER, or DL_FLAG_AUTOPROBE_CONSUMER set inflags atthe same time is invalid and will cause NULL to be returned upfront.However, if a device link between the givenconsumer andsupplier pairexists already when this function is called for them, the existing link willbe returned regardless of its current type and status (the link’s flags maybe modified then). The caller of this function is then expected to treatthe link as though it has just been created, so (in particular) ifDL_FLAG_STATELESS was passed inflags, the link needs to be releasedexplicitly when not needed any more (as stated above).

A side effect of the link creation is re-ordering of dpm_list and thedevices_kset list by moving the consumer device and all devices dependingon it to the ends of these lists (that does not happen to devices that havenot been registered when this function is called).

The supplier device is required to be registered when this function is calledand NULL will be returned if that is not the case. The consumer device neednot be registered, however.

voiddevice_link_del(structdevice_link * link)¶

Delete a stateless link between two devices.

Parameters

structdevice_link*link

Device link to delete.

Description

The caller must ensure proper synchronization of this function with runtimePM. If the link was added multiple times, it needs to be deleted as often.Care is required for hotplugged devices: Their links are purged on removaland callingdevice_link_del() is then no longer allowed.

voiddevice_link_remove(void * consumer, structdevice * supplier)¶

Delete a stateless link between two devices.

Parameters

void*consumer

Consumer end of the link.

structdevice*supplier

Supplier end of the link.

Description

The caller must ensure proper synchronization of this function with runtimePM.