Initiator API and Open iSCSI Interface Module

• Open-iSCSI Interface Module (ocs_openiscsi)

Overview

The Emulex Open-iSCSI interface (ocs_openiscsi) module is a separate loadable module included in the OneCore Storage SDK package to be used for initiator support with the Linux ocs_iscsi_driver (see the driver/linux/ocs_openiscsi directory).

The ocs_openiscsi module acts as an interface adapter between the driver’s Initiator API and the Open-iSCSI kernel module (iSCSI transport and libiscsi.ko). The ocs_openiscsi module registers with Linux ocs_iscsi_ramd driver as an initiator, then registers with Open-iSCSI as a transport. Emulex SLI-4 adapters are then presented to the SCSI Mid-layer as available SCSI host devices.

The ocs_openiscsi module depends on symbols exported from the Open-iSCSI’s libiscsi.ko module (libiscsi) and the Linux ocs_iscsi_ramd driver. As such, the libiscsi and Linux ocs_iscsi_ramd driver must be loaded before the ocs_openiscsi module is loaded.

Once the ocs_openiscsi module is loaded, it calls into the libiscsi to register itself as an interface and calls into the Linux ocs_iscsi_ramd driver to register itself as an initiator.

Initiator Task Tag (ITT) Handling

Open-iSCSI presents its iSCSI transport with complete PDUs to be executed. The ITT is set by the libiscsi and is used on a task completion to associate the response PDU with the correct task.

SLI-4 has its own rules about the contents of the ITT in a PDU. The ITT for SLI-4 must contain the WRB index and the SGL_ICD index for the command being submitted. To accommodate these requirements, the ocs_openiscsi module must store the ITT when a task is submitted, and then restore the ITT before it reports a completion back to the libiscsi.ko. The ITT is saved by the ocs_alloc_pdu() function, and then restored by the various callback functions.

ocs_openiscsi Module Load

When the ocs_openiscsi module is loaded, it calls iscsi_register_transport. This provides the libiscsi with pointers (such as start_conn, ep_connect, and get_stats) to the ocs_openiscsi module functions (such as ocs_conn_start(), ocs_ep_connect(), and ocs_conn_get_stats()) that are needed to act as a transport. The libiscsi can then use those ocs_openiscsi module functions to manage Open-iSCSI connections and I/Os.

After registering with the libiscsi, the ocs_openiscsi module calls ocs_get_instance() in the Linux ocs_iscsi_ramd driver to get information about each device that the driver has available. For each of these devices, the ocs_openiscsi module calls ocs_scsi_register_ini_templ() in the driver. This function call provides pointers to functions that the driver can use to add/delete domains, sports, and targets, and to send notifications to the ocs_openiscsi module.

Finally, the ocs_openiscsi module completes the registration with the libiscsi by calling iscsi_host_add and iscsi_create_iface.

ocs_openiscsi Module Load

The connect process of Open-iSCSI to the Linux ocs_iscsi_ramd driver has four steps:

1. Create an endpoint.
2. Create a session.
3. Create a connection structure.
4. Bind the three (endpoint, session, and connection structure) together.

For the first step, the Open-iSCSI iSCSI transport calls ocs_ep_connect(). The ocs_ep_connect() function calls into the libiscsi to create the endpoint structure, then calls ocs_xport_control() into the Linux ocs_iscsi_ramd driver to establish the connection.

The remaining three steps do not require any interaction with the Linux ocs_iscsi_ramd driver. The iSCSI transport calls into the ocs_openiscsi module, which calls into libiscsi to create the required structures and bind them.

Management PDU Processing

Open-iSCSI has an interface to send management (non-I/O) PDUs, such as LOGIN and NO-OP.

The iSCSI transport calls iscsi_conn_send_pdu, which then calls ocs_alloc_pdu() to allocate memory for the transaction. It fills in the PDU structure and calls ocs_task_xmit(). Within the task structure, the scsi_cmnd pointer is NULL (the distinguishing factor between a management task and an I/O task). The ocs_task_xmit() keys off the NULL scsi_cmnd pointer and calls ocs_xmit_mtask() to send the management task. The ocs_xmit_mtask() function sets up the correct callback function based on the type of PDU, and then calls into ocs_iscsi_unassisted_pdu_send() in the ocs_iscsi_ramd driver’s iSCSI API to send the PDU.

Depending on the value of the set_dm argument, the callback is called either when the PDU is sent or when the response to the PDU is received. The callback function performs any required processing on the response; calls iscsi_complete_pdu to notify libiscsi that the task is done; and finally calls ocs_scsi_io_free() to free the I/O resources.

IO PDU Processing

I/O PDU processing is similar to management PDU processing. The I/O begins in the SCSI mid-layer with a call to scsi_dispatch_cmd, which then gets forwarded to iscsi_queue_command in libiscsi. A PDU is allocated by calling ocs_alloc_pdu(). The libiscsi fills in the PDU details, including the SCSI CDB and data; and then calls ocs_task_xmit() to send the task.

The ocs_openiscsi module identifies the task as a SCSI command task due to the non-NULL scsi_cmd pointer. Based on the sc_data_direction field in the scsi_cmd, the ocs_openiscsi module calls ocs_scsi_send_rd_io() or ocs_scsi_send_wr_io(). The command callback is set to ocs_lnx_xport_io_cb().

When the command completes and ocs_lnx_xport_io_cb() is called, the ocs_openiscsi module calls iscsi_complete_scsi_task to inform the libiscsi that the task is complete. Next, the ocs_openiscsi module calls ocs_scsi_cmd_cb(), which then calls the scsi_done function from the I/O.

Finally, the ocs_openiscsi module calls ocs_scsi_io_free to release the I/O resources.