General Information¶

The following chapters describe the I/O related interface routines theLinux/390 common device support (CDS) provides to allow for device specificdriver implementations on the IBM ESA/390 hardware platform. Those interfacesintend to provide the functionality required by every device driverimplementation to allow to drive a specific hardware device on the ESA/390platform. Some of the interface routines are specific to Linux/390 and someof them can be found on other Linux platforms implementations too.Miscellaneous function prototypes, data declarations, and macro definitionscan be found in the architecture specific C header filelinux/arch/s390/include/asm/irq.h.

Overview of CDS interface concepts¶

Different to other hardware platforms, the ESA/390 architecture doesn’t defineinterrupt lines managed by a specific interrupt controller and bus systemsthat may or may not allow for shared interrupts, DMA processing, etc.. Instead,the ESA/390 architecture has implemented a so called channel subsystem, thatprovides a unified view of the devices physically attached to the systems.Though the ESA/390 hardware platform knows about a huge variety of differentperipheral attachments like disk devices (aka. DASDs), tapes, communicationcontrollers, etc. they can all be accessed by a well defined access method andthey are presenting I/O completion a unified way : I/O interruptions. Everysingle device is uniquely identified to the system by a so called subchannel,where the ESA/390 architecture allows for 64k devices be attached.

Linux, however, was first built on the Intel PC architecture, with its twocascaded 8259 programmable interrupt controllers (PICs), that allow for amaximum of 15 different interrupt lines. All devices attached to such a systemshare those 15 interrupt levels. Devices attached to the ISA bus system mustnot share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggeredinterrupts. MCA, EISA, PCI and other bus systems base on level triggeredinterrupts, and therewith allow for shared IRQs. However, if multiple devicespresent their hardware status by the same (shared) IRQ, the operating systemhas to call every single device driver registered on this IRQ in order todetermine the device driver owning the device that raised the interrupt.

Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).For internal use of the common I/O layer, these are still there. However,device drivers should use the new calling interface via the ccw_device only.

During its startup the Linux/390 system checks for peripheral devices. Eachof those devices is uniquely defined by a so called subchannel by the ESA/390channel subsystem. While the subchannel numbers are system generated, eachsubchannel also takes a user defined attribute, the so called device number.Both subchannel number and device number cannot exceed 65535. During sysfsinitialisation, the information about control unit type and device types thatimply specific I/O commands (channel command words - CCWs) in order to operatethe device are gathered. Device drivers can retrieve this set of hardwareinformation during their initialization step to recognize the devices theysupport using the information saved in the struct ccw_device given to them.This methods implies that Linux/390 doesn’t require to probe for free (notarmed) interrupt request lines (IRQs) to drive its devices with. Whereapplicable, the device drivers can use issue the READ DEVICE CHARACTERISTICSccw to retrieve device characteristics in its online routine.

In order to allow for easy I/O initiation the CDS layer provides accw_device_start() interface that takes a device specific channel program (oneor more CCWs) as input sets up the required architecture specific control blocksand initiates an I/O request on behalf of the device driver. Theccw_device_start() routine allows to specify whether it expects the CDS layerto notify the device driver for every interrupt it observes, or with final statusonly. Seeccw_device_start() for more details. A device driver must never issueESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.

For long running I/O request to be canceled, the CDS layer provides theccw_device_halt() function. Some devices require to initially issue a HALTSUBCHANNEL (HSCH) command without having pending I/O requests. This function isalso covered byccw_device_halt().

get_ciw() - get command information word

This call enables a device driver to get information about supported commandsfrom the extended SenseID data.

struct ciw *ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);

ccw_device_get_ciw() returns:

ccw_device_start() - Initiate I/O Request

Theccw_device_start() routines is the I/O request front-end processor. Alldevice driver I/O requests must be issued using this routine. A device drivermust not issue ESA/390 I/O commands itself. Instead theccw_device_start()routine provides all interfaces required to drive arbitrary devices.

This description also covers the status information passed to the devicedriver’s interrupt handler as this is related to the rules (flags) definedwith the associated I/O request when callingccw_device_start().

int ccw_device_start(struct ccw_device *cdev,                     struct ccw1 *cpa,                     unsigned long intparm,                     __u8 lpm,                     unsigned long flags);int ccw_device_start_timeout(struct ccw_device *cdev,                             struct ccw1 *cpa,                             unsigned long intparm,                             __u8 lpm,                             unsigned long flags,                             int expires);int ccw_device_start_key(struct ccw_device *cdev,                         struct ccw1 *cpa,                         unsigned long intparm,                         __u8 lpm,                         __u8 key,                         unsigned long flags);int ccw_device_start_key_timeout(struct ccw_device *cdev,                                 struct ccw1 *cpa,                                 unsigned long intparm,                                 __u8 lpm,                                 __u8 key,                                 unsigned long flags,                                 int expires);

Possible flag values are:

The cpa parameter points to the first format 1 CCW of a channel program:

struct ccw1 {      __u8  cmd_code;/* command code */      __u8  flags;   /* flags, like IDA addressing, etc. */      __u16 count;   /* byte count */      __u32 cda;     /* data address */} __attribute__ ((packed,aligned(8)));

with the following CCW flags values defined:

Viaccw_device_set_options(), the device driver may specify the followingoptions for the device:

Theccw_device_start() function returns:

When the I/O request completes, the CDS first level interrupt handler willaccumulate the status in a struct irb and then call the device interrupt handler.The intparm field will contain the value the device driver has associated with aparticular I/O request. If a pending device status was recognized,intparm will be set to 0 (zero). This may happen during I/O initiation or delayedby an alert status notification. In any case this status is not related to thecurrent (last) I/O request. In case of a delayed status notification no specialinterrupt will be presented to indicate I/O completion as the I/O request wasnever started, even thoughccw_device_start() returned with successful completion.

The irb may contain an error value, and the device driver should check for thisfirst:

If the concurrent sense flag in the extended status word (esw) in the irb isset, the field erw.scnt in the esw describes the number of device specificsense bytes available in the extended control word irb->scsw.ecw[]. No devicesensing by the device driver itself is required.

The device interrupt handler can use the following definitions to investigatethe primary unit check source coded in sense byte 0 :

Depending on the device status, multiple of those values may be set together.Please refer to the device specific documentation for details.

The irb->scsw.cstat field provides the (accumulated) subchannel status :

The irb->scsw.dstat field provides the (accumulated) device status :

Please see the ESA/390 Principles of Operation manual for details on theindividual flag meanings.

Usage Notes:

ccw_device_start() must be called disabled and with the ccw device lock held.

The device driver is allowed to issue the nextccw_device_start() call fromwithin its interrupt handler already. It is not required to schedule abottom-half, unless a non deterministically long running error recovery procedureor similar needs to be scheduled. During I/O processing the Linux/390 genericI/O device driver support has already obtained the IRQ lock, i.e. the handlermust not try to obtain it again when callingccw_device_start() or we end in adeadlock situation!

If a device driver relies on an I/O request to be completed prior to start thenext it can reduce I/O processing overhead by chaining a NoOp I/O commandCCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-Endand Device-End status to be presented together, with a single interrupt.However, this should be used with care as it implies the channel will remainbusy, not being able to process I/O requests for other devices on the samechannel. Therefore e.g. read commands should never use this technique, as theresult will be presented by a single interrupt anyway.

In order to minimize I/O overhead, a device driver should use theDOIO_REPORT_ALL only if the device can report intermediate interruptinformation prior to device-end the device driver urgently relies on. In thiscase all I/O interruptions are presented to the device driver until finalstatus is recognized.

If a device is able to recover from asynchronously presented I/O errors, it canperform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While somedevices always report channel-end and device-end together, with a singleinterrupt, others present primary status (channel-end) when the channel isready for the next I/O request and secondary status (device-end) when the datatransmission has been completed at the device.

Above flag allows to exploit this feature, e.g. for communication devices thatcan handle lost data on the network to allow for enhanced I/O processing.

Unless the channel subsystem at any time presents a secondary status interrupt,exploiting this feature will cause only primary status interrupts to bepresented to the device driver while overlapping I/O is performed. When asecondary status without error (alert status) is presented, this indicatessuccessful completion for all overlappingccw_device_start() requests that havebeen issued since the last secondary (final) status.

Channel programs that intend to set the suspend flag on a channel command word(CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or thesuspend flag will cause a channel program check. At the time the channel programbecomes suspended an intermediate interrupt will be generated by the channelsubsystem.

ccw_device_resume() - Resume Channel Program Execution

If a device driver chooses to suspend the current channel program execution bysetting the CCW suspend flag on a particular CCW, the channel program executionis suspended. In order to resume channel program execution the CIO layerprovides theccw_device_resume() routine.

int ccw_device_resume(struct ccw_device *cdev);

Theccw_device_resume() function returns:

Usage Notes:

Please have a look at theccw_device_start() usage notes for more details onsuspended channel programs.

ccw_device_halt() - Halt I/O Request Processing

Sometimes a device driver might need a possibility to stop the processing ofa long-running channel program or the device might require to initially issuea halt subchannel (HSCH) I/O command. For those purposes theccw_device_halt()command is provided.

ccw_device_halt() must be called disabled and with the ccw device lock held.

int ccw_device_halt(struct ccw_device *cdev,                    unsigned long intparm);

Theccw_device_halt() function returns:

Usage Notes:

A device driver may write a never-ending channel program by writing a channelprogram that at its end loops back to its beginning by means of a transfer inchannel (TIC) command (CCW_CMD_TIC). Usually this is performed by networkdevice drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW isexecuted a program controlled interrupt (PCI) is generated. The device drivercan then perform an appropriate action. Prior to interrupt of an outstandingread to a network device (with or without PCI flag) accw_device_halt()is required to end the pending operation.

ccw_device_clear() - Terminage I/O Request Processing

In order to terminate all I/O processing at the subchannel, the clear subchannel(CSCH) command is used. It can be issued viaccw_device_clear().

ccw_device_clear() must be called disabled and with the ccw device lock held.

int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);

Theccw_device_clear() function returns:

Miscellaneous Support Routines¶

This chapter describes various routines to be used in a Linux/390 devicedriver programming environment.

get_ccwdev_lock()

Get the address of the device specific lock. This is then used inspin_lock() / spin_unlock() calls.

__u8 ccw_device_get_path_mask(struct ccw_device *cdev);

Get the mask of the path currently available for cdev.