Introduction¶

Modern SoCs typically have heterogeneous remote processor devices in asymmetricmultiprocessing (AMP) configurations, which may be running different instancesof operating system, whether it’s Linux or any other flavor of real-time OS.

OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP.In a typical configuration, the dual cortex-A9 is running Linux in a SMPconfiguration, and each of the other three cores (two M3 cores and a DSP)is running its own instance of RTOS in an AMP configuration.

The remoteproc framework allows different platforms/architectures tocontrol (power on, load firmware, power off) those remote processors whileabstracting the hardware differences, so the entire driver doesn’t need to beduplicated. In addition, this framework also adds rpmsg virtio devicesfor remote processors that supports this kind of communication. This way,platform-specific remoteproc drivers only need to provide a few low-levelhandlers, and then all rpmsg drivers will then just work(for more information about the virtio-based rpmsg bus and its drivers,please readRemote Processor Messaging (rpmsg) Framework).Registration of other types of virtio devices is now also possible. Firmwaresjust need to publish what kind of virtio devices do they support, and thenremoteproc will add those devices. This makes it possible to reuse theexisting virtio drivers with remote processor backends at a minimal developmentcost.

User API¶

int rproc_boot(struct rproc *rproc)

Boot a remote processor (i.e. load its firmware, power it on, ...).

If the remote processor is already powered on, this function immediatelyreturns (successfully).

Returns 0 on success, and an appropriate error value otherwise.Note: to use this function you should already have a valid rprochandle. There are several ways to achieve that cleanly (devres, pdata,the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, wemight also consider using dev_archdata for this).

int rproc_shutdown(struct rproc *rproc)

Power off a remote processor (previously booted withrproc_boot()).In case @rproc is still being used by an additional user(s), thenthis function will just decrement the power refcount and exit,without really powering off the device.

Returns 0 on success, and an appropriate error value otherwise.Every call torproc_boot() must (eventually) be accompanied by a calltorproc_shutdown(). Callingrproc_shutdown() redundantly is a bug.

struct rproc *rproc_get_by_phandle(phandle phandle)

Find an rproc handle using a device tree phandle. Returns the rprochandle on success, and NULL on failure. This function incrementsthe remote processor’s refcount, so always userproc_put() todecrement it back once rproc isn’t needed anymore.

Typical usage¶

#include <linux/remoteproc.h>/* in case we were given a valid 'rproc' handle */int dummy_rproc_example(struct rproc *my_rproc){      int ret;      /* let's power on and boot our remote processor */      ret = rproc_boot(my_rproc);      if (ret) {              /*               * something went wrong. handle it and leave.               */      }      /*       * our remote processor is now powered on... give it some work       */      /* let's shut it down now */      rproc_shutdown(my_rproc);}

API for implementers¶

struct rproc *rproc_alloc(struct device *dev, const char *name,                              const struct rproc_ops *ops,                              const char *firmware, int len)

Allocate a new remote processor handle, but don’t registerit yet. Required parameters are the underlying device, thename of this remote processor, platform-specific ops handlers,the name of the firmware to boot this rproc with, and thelength of private data needed by the allocating rproc driver (in bytes).

This function should be used by rproc implementations duringinitialization of the remote processor.

After creating an rproc handle using this function, and when ready,implementations should then callrproc_add() to completethe registration of the remote processor.

On success, the new rproc is returned, and on failure, NULL.

void rproc_free(struct rproc *rproc)

Free an rproc handle that was allocated by rproc_alloc.

This function essentially unrollsrproc_alloc(), by decrementing therproc’s refcount. It doesn’t directly free rproc; that would happenonly if there are no other references to rproc and its refcount nowdropped to zero.

int rproc_add(struct rproc *rproc)

Register @rproc with the remoteproc framework, after it has beenallocated withrproc_alloc().

This is called by the platform-specific rproc implementation, whenevera new remote processor device is probed.

Returns 0 on success and an appropriate error code otherwise.Note: this function initiates an asynchronous firmware loadingcontext, which will look for virtio devices supported by the rproc’sfirmware.

If found, those virtio devices will be created and added, so as a resultof registering this remote processor, additional virtio drivers might getprobed.

int rproc_del(struct rproc *rproc)

Unrollrproc_add().

This function should be called when the platform specific rprocimplementation decides to remove the rproc device. it should_only_ be called if a previous invocation ofrproc_add()has completed successfully.

Afterrproc_del() returns, @rproc is still valid, and itslast refcount should be decremented by callingrproc_free().

Returns 0 on success and -EINVAL if @rproc isn’t valid.

void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type)

Report a crash in a remoteproc

This function must be called every time a crash is detected by theplatform specific rproc implementation. This should not be called from anon-remoteproc driver. This function can be called from atomic/interruptcontext.

Implementation callbacks¶

These callbacks should be provided by platform-specific remoteprocdrivers:

/** * struct rproc_ops - platform-specific device handlers * @start:    power on the device and boot it * @stop:     power off the device * @kick:     kick a virtqueue (virtqueue id given as a parameter) */struct rproc_ops {      int (*start)(struct rproc *rproc);      int (*stop)(struct rproc *rproc);      void (*kick)(struct rproc *rproc, int vqid);};

Every remoteproc implementation should at least provide the ->start and ->stophandlers. If rpmsg/virtio functionality is also desired, then the ->kick handlershould be provided as well.

The ->start() handler takes an rproc handle and should then power on thedevice and boot it (use rproc->priv to access platform-specific private data).The boot address, in case needed, can be found in rproc->bootaddr (remoteproccore puts there the ELF entry point).On success, 0 should be returned, and on failure, an appropriate error code.

The ->stop() handler takes an rproc handle and powers the device down.On success, 0 is returned, and on failure, an appropriate error code.

The ->kick() handler takes an rproc handle, and an index of a virtqueuewhere new message was placed in. Implementations should interrupt the remoteprocessor and let it know it has pending messages. Notifying remote processorsthe exact virtqueue index to look in is optional: it is easy (and nottoo expensive) to go through the existing virtqueues and look for new buffersin the used rings.

Binary Firmware Structure¶

At this point remoteproc supports ELF32 and ELF64 firmware binaries. However,it is quite expected that other platforms/devices which we’d want tosupport with this framework will be based on different binary formats.

When those use cases show up, we will have to decouple the binary formatfrom the framework core, so we can support several binary formats withoutduplicating common code.

When the firmware is parsed, its various segments are loaded to memoryaccording to the specified device address (might be a physical addressif the remote processor is accessing memory directly).

In addition to the standard ELF segments, most remote processors wouldalso include a special section which we call “the resource table”.

The resource table contains system resources that the remote processorrequires before it should be powered on, such as allocation of physicallycontiguous memory, or iommu mapping of certain on-chip peripherals.Remotecore will only power up the device after all the resource table’srequirement are met.

In addition to system resources, the resource table may also containresource entries that publish the existence of supported featuresor configurations by the remote processor, such as trace buffers andsupported virtio devices (and their configurations).

The resource table begins with this header:

/** * struct resource_table - firmware resource table header * @ver: version number * @num: number of resource entries * @reserved: reserved (must be zero) * @offset: array of offsets pointing at the various resource entries * * The header of the resource table, as expressed by this structure, * contains a version number (should we need to change this format in the * future), the number of available resource entries, and their offsets * in the table. */struct resource_table {      u32 ver;      u32 num;      u32 reserved[2];      u32 offset[0];} __packed;

Immediately following this header are the resource entries themselves,each of which begins with the following resource entry header:

/** * struct fw_rsc_hdr - firmware resource entry header * @type: resource type * @data: resource data * * Every resource entry begins with a 'struct fw_rsc_hdr' header providing * its @type. The content of the entry itself will immediately follow * this header, and it should be parsed according to the resource type. */struct fw_rsc_hdr {      u32 type;      u8 data[0];} __packed;

Some resources entries are mere announcements, where the host is informedof specific remoteproc configuration. Other entries require the host todo something (e.g. allocate a system resource). Sometimes a negotiationis expected, where the firmware requests a resource, and once allocated,the host should provide back its details (e.g. address of an allocatedmemory region).

Here are the various resource types that are currently supported:

/** * enum fw_resource_type - types of resource entries * * @RSC_CARVEOUT:   request for allocation of a physically contiguous *                memory region. * @RSC_DEVMEM:     request to iommu_map a memory-based peripheral. * @RSC_TRACE:            announces the availability of a trace buffer into which *                the remote processor will be writing logs. * @RSC_VDEV:       declare support for a virtio device, and serve as its *                virtio header. * @RSC_LAST:       just keep this one at the end * @RSC_VENDOR_START: start of the vendor specific resource types range * @RSC_VENDOR_END:   end of the vendor specific resource types range * * Please note that these values are used as indices to the rproc_handle_rsc * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to * check the validity of an index before the lookup table is accessed, so * please update it as needed. */enum fw_resource_type {      RSC_CARVEOUT            = 0,      RSC_DEVMEM              = 1,      RSC_TRACE               = 2,      RSC_VDEV                = 3,      RSC_LAST                = 4,      RSC_VENDOR_START        = 128,      RSC_VENDOR_END          = 512,};

For more details regarding a specific resource type, please see itsdedicated structure in include/linux/remoteproc.h.

We also expect that platform-specific resource entries will show upat some point. When that happens, we could easily add a new RSC_PLATFORMtype, and hand those resources to the platform-specific rproc driver to handle.

Virtio and remoteproc¶

The firmware should provide remoteproc information about virtio devicesthat it supports, and their configurations: a RSC_VDEV resource entryshould specify the virtio device id (as in virtio_ids.h), virtio features,virtio config space, vrings information, etc.

When a new remote processor is registered, the remoteproc frameworkwill look for its resource table and will register the virtio devicesit supports. A firmware may support any number of virtio devices, andof any type (a single remote processor can also easily support severalrpmsg virtio devices this way, if desired).

Of course, RSC_VDEV resource entries are only good enough for staticallocation of virtio devices. Dynamic allocations will also be made possibleusing the rpmsg bus (similar to how we already do dynamic allocations ofrpmsg channels; read more about it inRemote Processor Messaging (rpmsg) Framework).