target and iSCSI Interfaces Guide

Introduction and Overview

TBD

Target core device interfaces

This section is blank because no kerneldoc comments have been added todrivers/target/target_core_device.c.

Target core transport interfaces

voidtransport_init_session(structse_session*se_sess)

initialize a session object

Parameters

structse_session*se_sess

Session object pointer.

Description

The caller must have zero-initializedse_sess before calling this function.

structse_session*transport_alloc_session(enumtarget_prot_opsup_prot_ops)

allocate a session object and initialize it

Parameters

enumtarget_prot_opsup_prot_ops

bitmask that defines which T10-PI modes are supported.

inttransport_alloc_session_tags(structse_session*se_sess,unsignedinttag_num,unsignedinttag_size)

allocate target driver private data

Parameters

structse_session*se_sess

Session pointer.

unsignedinttag_num

Maximum number of in-flight commands between initiator and target.

unsignedinttag_size

Size in bytes of the private data a target driver associates witheach command.

inttarget_init_cmd(structse_cmd*se_cmd,structse_session*se_sess,unsignedchar*sense,u64unpacked_lun,u32data_length,inttask_attr,intdata_dir,intflags)

initialize se_cmd

Parameters

structse_cmd*se_cmd

command descriptor to init

structse_session*se_sess

associated se_sess for endpoint

unsignedchar*sense

pointer to SCSI sense buffer

u64unpacked_lun

unpacked LUN to reference forstructse_lun

u32data_length

fabric expected data transfer length

inttask_attr

SAM task attribute

intdata_dir

DMA data direction

intflags

flags for command submission from target_sc_flags_tables

Description

Task tags are supported if the caller has setse_cmd->tag.

If the fabric driver calls target_stop_session, then it must check thereturn code and handle failures. This will never fail for other drivers,and the return code can be ignored.

Return

  • less than zero to signal active I/O shutdown failure.

  • zero on success.

inttarget_submit_prep(structse_cmd*se_cmd,unsignedchar*cdb,structscatterlist*sgl,u32sgl_count,structscatterlist*sgl_bidi,u32sgl_bidi_count,structscatterlist*sgl_prot,u32sgl_prot_count,gfp_tgfp)

prepare cmd for submission

Parameters

structse_cmd*se_cmd

command descriptor to prep

unsignedchar*cdb

pointer to SCSI CDB

structscatterlist*sgl

structscatterlist memory for unidirectional mapping

u32sgl_count

scatterlist count for unidirectional mapping

structscatterlist*sgl_bidi

structscatterlist memory for bidirectional READ mapping

u32sgl_bidi_count

scatterlist count for bidirectional READ mapping

structscatterlist*sgl_prot

structscatterlist memory protection information

u32sgl_prot_count

scatterlist count for protection information

gfp_tgfp

gfp allocation type

Return

  • less than zero to signal failure.

  • zero on success.

Description

If failure is returned, lio will the callers queue_status to completethe cmd.

voidtarget_submit_cmd(structse_cmd*se_cmd,structse_session*se_sess,unsignedchar*cdb,unsignedchar*sense,u64unpacked_lun,u32data_length,inttask_attr,intdata_dir,intflags)

lookup unpacked lun and submit uninitialized se_cmd

Parameters

structse_cmd*se_cmd

command descriptor to submit

structse_session*se_sess

associated se_sess for endpoint

unsignedchar*cdb

pointer to SCSI CDB

unsignedchar*sense

pointer to SCSI sense buffer

u64unpacked_lun

unpacked LUN to reference forstructse_lun

u32data_length

fabric expected data transfer length

inttask_attr

SAM task attribute

intdata_dir

DMA data direction

intflags

flags for command submission from target_sc_flags_tables

Description

Task tags are supported if the caller has setse_cmd->tag.

This may only be called from process context, and also currentlyassumes internal allocation of fabric payload buffer by target-core.

It also assumes interal target core SGL memory allocation.

This function must only be used by drivers that do their ownsync during shutdown and does not use target_stop_session. If thereis a failure this function will call into the fabric driver’squeue_status with a CHECK_CONDITION.

inttarget_submit(structse_cmd*se_cmd)

perform final initialization and submit cmd to LIO core

Parameters

structse_cmd*se_cmd

command descriptor to submit

Description

target_submit_prep or something similar must have been called on the cmd,and this must be called from process context.

inttarget_submit_tmr(structse_cmd*se_cmd,structse_session*se_sess,unsignedchar*sense,u64unpacked_lun,void*fabric_tmr_ptr,unsignedchartm_type,gfp_tgfp,u64tag,intflags)

lookup unpacked lun and submit uninitialized se_cmd for TMR CDBs

Parameters

structse_cmd*se_cmd

command descriptor to submit

structse_session*se_sess

associated se_sess for endpoint

unsignedchar*sense

pointer to SCSI sense buffer

u64unpacked_lun

unpacked LUN to reference forstructse_lun

void*fabric_tmr_ptr

fabric context for TMR req

unsignedchartm_type

Type of TM request

gfp_tgfp

gfp type for caller

u64tag

referenced task tag for TMR_ABORT_TASK

intflags

submit cmd flags

Description

Callable from all contexts.

inttarget_get_sess_cmd(structse_cmd*se_cmd,boolack_kref)

Verify the session is accepting cmds and take ref

Parameters

structse_cmd*se_cmd

command descriptor to add

boolack_kref

Signal that fabric will perform an acktarget_put_sess_cmd()

inttarget_put_sess_cmd(structse_cmd*se_cmd)

decrease the command reference count

Parameters

structse_cmd*se_cmd

command to drop a reference from

Description

Returns 1 if and only if thistarget_put_sess_cmd() call caused therefcount to drop to zero. Returns zero otherwise.

voidtarget_stop_cmd_counter(structtarget_cmd_counter*cmd_cnt)

Stop new IO from being added to the counter.

Parameters

structtarget_cmd_counter*cmd_cnt

counter to stop

voidtarget_stop_session(structse_session*se_sess)

Stop new IO from being queued on the session.

Parameters

structse_session*se_sess

session to stop

voidtarget_wait_for_cmds(structtarget_cmd_counter*cmd_cnt)

Wait for outstanding cmds.

Parameters

structtarget_cmd_counter*cmd_cnt

counter to wait for active I/O for.

voidtarget_wait_for_sess_cmds(structse_session*se_sess)

Wait for outstanding commands

Parameters

structse_session*se_sess

session to wait for active I/O

booltransport_wait_for_tasks(structse_cmd*cmd)

set CMD_T_STOP and wait for t_transport_stop_comp

Parameters

structse_cmd*cmd

command to wait on

inttarget_send_busy(structse_cmd*cmd)

Send SCSI BUSY status back to the initiator

Parameters

structse_cmd*cmd

SCSI command for which to send a BUSY reply.

Note

Only call this function if target_submit_cmd*() failed.

Target-supported userspace I/O

Userspace I/O

Define a shared-memory interface for LIO to pass SCSI commands anddata to userspace for processing. This is to allow backends thatare too complex for in-kernel support to be possible.

It uses the UIO framework to do a lot of the device-creation andintrospection work for us.

See the .h file for how the ring is laid out. Note that while thecommand ring is defined, the particulars of the data area arenot. Offset values in the command entry point to other locationsinternal to the mmap-ed area. There is separate space outside thecommand ring for data buffers. This leaves maximum flexibility formoving buffer allocations, or even page flipping or otherallocation techniques, without altering the command ring layout.

SECURITY:The user process must be assumed to be malicious. There’s no way toprevent it breaking the command ring protocol if it wants, but inorder to prevent other issues we must only ever readdata fromthe shared memory area, not offsets or sizes. This applies tocommand ring entries as well as the mailbox. Extra code needed forthis may have a ‘UAM’ comment.

Ring Design

The mmaped area is divided into three parts:1) The mailbox (structtcmu_mailbox, below);2) The command ring;3) Everything beyond the command ring (data).

The mailbox tells userspace the offset of the command ring from thestart of the shared memory region, and how big the command ring is.

The kernel passes SCSI commands to userspace by putting astructtcmu_cmd_entry in the ring, updating mailbox->cmd_head, and pokinguserspace via UIO’s interrupt mechanism.

tcmu_cmd_entry contains a header. If the header type is PAD,userspace should skip hdr->length bytes (mod cmdr_size) to find thenext cmd_entry.

Otherwise, the entry will contain offsets into the mmaped area thatcontain the cdb and data buffers -- the latter accessible via theiov array. iov addresses are also offsets into the shared area.

When userspace is completed handling the command, setentry->rsp.scsi_status, fill in rsp.sense_buffer if appropriate,and also set mailbox->cmd_tail equal to the old cmd_tail plushdr->length, mod cmdr_size. If cmd_tail doesn’t equal cmd_head, itshould process the next packet the same way, and so on.

iSCSI helper functions

voidiscsi_prep_data_out_pdu(structiscsi_task*task,structiscsi_r2t_info*r2t,structiscsi_data*hdr)

initialize Data-Out

Parameters

structiscsi_task*task

scsi command task

structiscsi_r2t_info*r2t

R2T info

structiscsi_data*hdr

iscsi data in pdu

Notes

Initialize Data-Out within this R2T sequence and findsproper data_offset within this SCSI command.

This function is called with connection lock taken.

void__iscsi_put_task(structiscsi_task*task)

drop the refcount on a task

Parameters

structiscsi_task*task

iscsi_task to drop the refcount on

Description

The back_lock must be held when calling in case it frees the task.

voidiscsi_complete_scsi_task(structiscsi_task*task,uint32_texp_cmdsn,uint32_tmax_cmdsn)

finish scsi task normally

Parameters

structiscsi_task*task

iscsi task for scsi cmd

uint32_texp_cmdsn

expected cmd sn in cpu format

uint32_tmax_cmdsn

max cmd sn in cpu format

Description

This is used when drivers do not need or cannot performlower level pdu processing.

Called with session back_lock

structiscsi_task*iscsi_itt_to_task(structiscsi_conn*conn,itt_titt)

look up task by itt

Parameters

structiscsi_conn*conn

iscsi connection

itt_titt

itt

Description

This should be used for mgmt tasks like login and nops, or ifthe LDD’s itt space does not include the session age.

The session back_lock must be held.

int__iscsi_complete_pdu(structiscsi_conn*conn,structiscsi_hdr*hdr,char*data,intdatalen)

complete pdu

Parameters

structiscsi_conn*conn

iscsi conn

structiscsi_hdr*hdr

iscsi header

char*data

data buffer

intdatalen

len of data buffer

Description

Completes pdu processing by freeing any resources allocated atqueuecommand or send generic. session back_lock must be held and verifyitt must have been called.

structiscsi_task*iscsi_itt_to_ctask(structiscsi_conn*conn,itt_titt)

look up ctask by itt

Parameters

structiscsi_conn*conn

iscsi connection

itt_titt

itt

Description

This should be used for cmd tasks.

The session back_lock must be held.

voidiscsi_requeue_task(structiscsi_task*task)

requeue task to run from session workqueue

Parameters

structiscsi_task*task

task to requeue

Description

Callers must have taken a ref to the task that is going to be requeued.

voidiscsi_suspend_queue(structiscsi_conn*conn)

suspend iscsi_queuecommand

Parameters

structiscsi_conn*conn

iscsi conn to stop queueing IO on

Description

This grabs the session frwd_lock to make sure no one is inxmit_task/queuecommand, and then sets suspend to preventnew commands from being queued. This only needs to be calledby offload drivers that need to sync a path like ep disconnectwith the iscsi_queuecommand/xmit_task. To start IO again libiscsiwill call iscsi_start_tx and iscsi_unblock_session when in FFP.

voidiscsi_suspend_tx(structiscsi_conn*conn)

suspend iscsi_data_xmit

Parameters

structiscsi_conn*conn

iscsi conn to stop processing IO on.

Description

This function sets the suspend bit to prevent iscsi_data_xmitfrom sending new IO, and if work is queued on the xmit threadit will wait for it to be completed.

voidiscsi_suspend_rx(structiscsi_conn*conn)

Prevent recvwork from running again.

Parameters

structiscsi_conn*conn

iscsi conn to stop.

voidiscsi_conn_unbind(structiscsi_cls_conn*cls_conn,boolis_active)

prevent queueing to conn.

Parameters

structiscsi_cls_conn*cls_conn

iscsi conn ep is bound to.

boolis_active

is the conn in use for boot or is this for EH/termination

Description

This must be called by drivers implementing the ep_disconnect callout.It disables queueing to the connection from libiscsi in preparation foran ep_disconnect call.

intiscsi_eh_session_reset(structscsi_cmnd*sc)

drop session and attempt relogin

Parameters

structscsi_cmnd*sc

scsi command

Description

This function will wait for a relogin, session termination fromuserspace, or a recovery/replacement timeout.

intiscsi_eh_recover_target(structscsi_cmnd*sc)

reset target and possibly the session

Parameters

structscsi_cmnd*sc

scsi command

Description

This will attempt to send a warm target reset. If that fails,we will escalate to ERL0 session recovery.

intiscsi_host_add(structScsi_Host*shost,structdevice*pdev)

add host to system

Parameters

structScsi_Host*shost

scsi host

structdevice*pdev

parent device

Description

This should be called by partial offload and software iscsi driversto add a host to the system.

structScsi_Host*iscsi_host_alloc(conststructscsi_host_template*sht,intdd_data_size,boolxmit_can_sleep)

allocate a host and driver data

Parameters

conststructscsi_host_template*sht

scsi host template

intdd_data_size

driver host data size

boolxmit_can_sleep

bool indicating if LLD will queue IO from a work queue

Description

This should be called by partial offload and software iscsi drivers.To access the driver specific memory use theiscsi_host_priv() macro.

voidiscsi_host_remove(structScsi_Host*shost,boolis_shutdown)

remove host and sessions

Parameters

structScsi_Host*shost

scsi host

boolis_shutdown

true if called from a driver shutdown callout

Description

If there are any sessions left, this will initiate the removal and waitfor the completion.

structiscsi_cls_session*iscsi_session_setup(structiscsi_transport*iscsit,structScsi_Host*shost,uint16_tcmds_max,intdd_size,intcmd_task_size,uint32_tinitial_cmdsn,unsignedintid)

create iscsi cls session and host and session

Parameters

structiscsi_transport*iscsit

iscsi transport template

structScsi_Host*shost

scsi host

uint16_tcmds_max

session can queue

intdd_size

private driver data size, added to session allocation size

intcmd_task_size

LLD task private data size

uint32_tinitial_cmdsn

initial CmdSN

unsignedintid

target ID to add to this session

Description

This can be used by software iscsi_transports that allocatea session per scsi host.

Callers should set cmds_max to the largest total numer (mgmt + scsi) oftasks they support. The iscsi layer reserves ISCSI_MGMT_CMDS_MAX tasksfor nop handling and login/logout requests.

voidiscsi_session_free(structiscsi_cls_session*cls_session)

Free iscsi session and it’s resources

Parameters

structiscsi_cls_session*cls_session

iscsi session

voidiscsi_session_teardown(structiscsi_cls_session*cls_session)

destroy session and cls_session

Parameters

structiscsi_cls_session*cls_session

iscsi session

structiscsi_cls_conn*iscsi_conn_setup(structiscsi_cls_session*cls_session,intdd_size,uint32_tconn_idx)

create iscsi_cls_conn and iscsi_conn

Parameters

structiscsi_cls_session*cls_session

iscsi_cls_session

intdd_size

private driver data size

uint32_tconn_idx

cid

voidiscsi_conn_teardown(structiscsi_cls_conn*cls_conn)

teardown iscsi connection

Parameters

structiscsi_cls_conn*cls_conn

iscsi class connection

Description

TODO: we may need to make this into a two step processlike scsi-mls remove + put host

iSCSI boot information

structiscsi_boot_kobj*iscsi_boot_create_target(structiscsi_boot_kset*boot_kset,intindex,void*data,ssize_t(*show)(void*data,inttype,char*buf),umode_t(*is_visible)(void*data,inttype),void(*release)(void*data))

create boot target sysfs dir

Parameters

structiscsi_boot_kset*boot_kset

boot kset

intindex

the target id

void*data

driver specific data for target

ssize_t(*show)(void*data,inttype,char*buf)

attr show function

umode_t(*is_visible)(void*data,inttype)

attr visibility function

void(*release)(void*data)

release function

Note

The boot sysfs lib will free the data passed in for the callerwhen all refs to the target kobject have been released.

structiscsi_boot_kobj*iscsi_boot_create_initiator(structiscsi_boot_kset*boot_kset,intindex,void*data,ssize_t(*show)(void*data,inttype,char*buf),umode_t(*is_visible)(void*data,inttype),void(*release)(void*data))

create boot initiator sysfs dir

Parameters

structiscsi_boot_kset*boot_kset

boot kset

intindex

the initiator id

void*data

driver specific data

ssize_t(*show)(void*data,inttype,char*buf)

attr show function

umode_t(*is_visible)(void*data,inttype)

attr visibility function

void(*release)(void*data)

release function

Note

The boot sysfs lib will free the data passed in for the callerwhen all refs to the initiator kobject have been released.

structiscsi_boot_kobj*iscsi_boot_create_ethernet(structiscsi_boot_kset*boot_kset,intindex,void*data,ssize_t(*show)(void*data,inttype,char*buf),umode_t(*is_visible)(void*data,inttype),void(*release)(void*data))

create boot ethernet sysfs dir

Parameters

structiscsi_boot_kset*boot_kset

boot kset

intindex

the ethernet device id

void*data

driver specific data

ssize_t(*show)(void*data,inttype,char*buf)

attr show function

umode_t(*is_visible)(void*data,inttype)

attr visibility function

void(*release)(void*data)

release function

Note

The boot sysfs lib will free the data passed in for the callerwhen all refs to the ethernet kobject have been released.

structiscsi_boot_kobj*iscsi_boot_create_acpitbl(structiscsi_boot_kset*boot_kset,intindex,void*data,ssize_t(*show)(void*data,inttype,char*buf),umode_t(*is_visible)(void*data,inttype),void(*release)(void*data))

create boot acpi table sysfs dir

Parameters

structiscsi_boot_kset*boot_kset

boot kset

intindex

not used

void*data

driver specific data

ssize_t(*show)(void*data,inttype,char*buf)

attr show function

umode_t(*is_visible)(void*data,inttype)

attr visibility function

void(*release)(void*data)

release function

Note

The boot sysfs lib will free the data passed in for the callerwhen all refs to the acpitbl kobject have been released.

structiscsi_boot_kset*iscsi_boot_create_kset(constchar*set_name)

creates root sysfs tree

Parameters

constchar*set_name

name of root dir

structiscsi_boot_kset*iscsi_boot_create_host_kset(unsignedinthostno)

creates root sysfs tree for a scsi host

Parameters

unsignedinthostno

host number of scsi host

voidiscsi_boot_destroy_kset(structiscsi_boot_kset*boot_kset)

destroy kset and kobjects under it

Parameters

structiscsi_boot_kset*boot_kset

boot kset

Description

This will remove the kset and kobjects and attrs under it.

iSCSI TCP interfaces

intiscsi_sw_tcp_recv(read_descriptor_t*rd_desc,structsk_buff*skb,unsignedintoffset,size_tlen)

TCP receive in sendfile fashion

Parameters

read_descriptor_t*rd_desc

read descriptor

structsk_buff*skb

socket buffer

unsignedintoffset

offset in skb

size_tlen

skb->len - offset

intiscsi_sw_sk_state_check(structsock*sk)

check socket state

Parameters

structsock*sk

socket

Description

If the socket is in CLOSE or CLOSE_WAIT we shouldnot close the connection if there is still somedata pending.

Must be called with sk_callback_lock.

voidiscsi_sw_tcp_write_space(structsock*sk)

Called when more output buffer space is available

Parameters

structsock*sk

socket space is available for

intiscsi_sw_tcp_xmit_segment(structiscsi_tcp_conn*tcp_conn,structiscsi_segment*segment)

transmit segment

Parameters

structiscsi_tcp_conn*tcp_conn

the iSCSI TCP connection

structiscsi_segment*segment

the buffer to transmnit

Description

This function transmits as much of the buffer asthe network layer will accept, and returns the number ofbytes transmitted.

If CRC hashing is enabled, the function will compute thehash as it goes. When the entire segment has been transmitted,it will retrieve the hash value and send it as well.

intiscsi_sw_tcp_xmit(structiscsi_conn*conn)

TCP transmit

Parameters

structiscsi_conn*conn

iscsi connection

intiscsi_sw_tcp_xmit_qlen(structiscsi_conn*conn)

return the number of bytes queued for xmit

Parameters

structiscsi_conn*conn

iscsi connection

intiscsi_tcp_segment_done(structiscsi_tcp_conn*tcp_conn,structiscsi_segment*segment,intrecv,unsignedcopied)

check whether the segment is complete

Parameters

structiscsi_tcp_conn*tcp_conn

iscsi tcp connection

structiscsi_segment*segment

iscsi segment to check

intrecv

set to one of this is called from the recv path

unsignedcopied

number of bytes copied

Description

Check if we’re done receiving this segment. If the receivebuffer is full but we expect more data, move on to thenext entry in the scatterlist.

If the amount of data we received isn’t a multiple of 4,we will transparently receive the pad bytes, too.

This function must be re-entrant.

voidiscsi_tcp_hdr_recv_prep(structiscsi_tcp_conn*tcp_conn)

prep segment for hdr reception

Parameters

structiscsi_tcp_conn*tcp_conn

iscsi connection to prep for

Description

This function always passes NULL for the crcp argument, because when thisfunction is called we do not yet know the final size of the header and wantto delay the digest processing until we know that.

voidiscsi_tcp_cleanup_task(structiscsi_task*task)

free tcp_task resources

Parameters

structiscsi_task*task

iscsi task

Description

must be called with session back_lock

intiscsi_tcp_recv_segment_is_hdr(structiscsi_tcp_conn*tcp_conn)

tests if we are reading in a header

Parameters

structiscsi_tcp_conn*tcp_conn

iscsi tcp conn

Description

returns non zero if we are currently processing or setup to processa header.

intiscsi_tcp_recv_skb(structiscsi_conn*conn,structsk_buff*skb,unsignedintoffset,booloffloaded,int*status)

Process skb

Parameters

structiscsi_conn*conn

iscsi connection

structsk_buff*skb

network buffer with header and/or data segment

unsignedintoffset

offset in skb

booloffloaded

bool indicating if transfer was offloaded

int*status

iscsi TCP status result

Description

Will return status of transfer instatus. And will returnnumber of bytes copied.

intiscsi_tcp_task_init(structiscsi_task*task)

Initialize iSCSI SCSI_READ or SCSI_WRITE commands

Parameters

structiscsi_task*task

scsi command task

intiscsi_tcp_task_xmit(structiscsi_task*task)

xmit normal PDU task

Parameters

structiscsi_task*task

iscsi command task

Description

We’re expected to return 0 when everything was transmitted successfully,-EAGAIN if there’s still data in the queue, or != 0 for any other kindof error.