Configuring iSCSI

iSCSI is an abbreviation of Internet Small Computer System Interface. iSCSI is simply a networked implementation of the well-known standardized SCSI protocol. SCSI defines a model, cabling, electrical signals and commands to use to access various devices. iSCSI uses just the model and the commands. SCSI (and hence: iSCSI) can be used to access all kinds of devices and though mostly used for disks and tape-units has been used to access USB storage, printers and flatbed scanners too, just to name a few.

The iSCSI setup is similar to the SCSI setup as it consists of a client and a server. In between is an IP based network, which can be a LAN, WAN or the Internet. The client issues low level SCSI commands as usual (e.g. read, write, seek or erase). The commands can be encrypted by the client and are then encapsulated and sent over the network. The server receives and possibly decrypts the commands and executes the commands. Unlike traditional Fibre Channel, which requires special-purpose cabling, iSCSI can be run over long distances using existing network infrastructure.

Clients are refered to as ’initiators’. The commands are referred to as ’CDBs’ (Command Descriptor Blocks) and the server storage devices are known as ’targets’.

Definitions:

iSCSI target iSCSI initiator

The exported storage entity is the target and the importing entity is the initiator.

4.2.3.1.1 iSCSI Initiator

On Linux hosts that act as a client (initiator) to a iSCSI server (target) you will need to install the client software. You will also need to edit the configuration file so you can discover the desired target(s). On Red Hat Linux and its derivatives the configuration file is /etc/iscsi/iscsid.conf. Edit the file so it points to the proper server (target) and contains a set of valid credentials - see example file at the bottom of this segment.

After configuration, ensure that the iSCSI service runs using the following command:

/etc/init.d/iscsi status

If needed start the iSCSI service using the following command:

/etc/init.d/iscsi start

Use the iscsiadm command to discover the targets. Where our example shows questionmarks, substitute the IP address of the target:

iscsiadm -m discovery -t sendtargets -p ???.???.???.???

After successfully discovering targets you can log in into the volumes that were discovered. This can be done with the iscsiadm tool, which can also be used to log out of a volume. A fast way to add newly discovered volumes is to restart the iscsi service:

/etc/init.d/iscsi restart

Tip

When adding an iscsi mount point to thefstabfile use the_netdevoption:

/dev/sdb1 /mnt/iscsi ext3(4) _netdev 0 0

The_netdevoption ensures that the network is up before trying to mount the device.

Example /etc/iscsi/iscsid.config:

################

# iSNS settings

################

# Address of iSNS server isns.address = ***.***.***.***

isns.port = 3260

# *************

# CHAP Settings

# *************

# To enable CHAP authentication set node.session.auth.authmethod

# to CHAP. The default is None.

node.session.auth.authmethod = CHAP

# To set a CHAP username and password for initiator

# authentication by the target(s), uncomment the following lines:

node.session.auth.username = ********

node.session.auth.password = ********

# To enable CHAP authentication for a discovery session to the target

# set discovery.sendtargets.auth.authmethod to CHAP. The default is None.

discovery.sendtargets.auth.authmethod = CHAP

# To set a discovery session CHAP username and password for the initiator

# authentication by the target(s), uncomment the following lines:

discovery.sendtargets.auth.username = ********

discovery.sendtargets.auth.password = ********

4.2.3.1.2 iSCSI Target

There are a number of ways to set up a target. The SCSI Target Framework (STGT/TGT) was the standard before linux 2.6.38.

The current standard is the LIO target.

Another well-known implementation was the iSCSI Enterprise Target (IET) and its successor the SCSI Target Subsystem (SCST).

IET was a candidate for kernel inclusion too, but LIO was the final choice. LIO (an abbreviation of linux-iscsi.org) now is the standard open-source SCSI Target for shared data storage in Linux. It supports all prevalent storage fabrics, including Fibre Channel (QLogic), FCoE, iEEE 1394, iSCSI, iSER (Mellanox InfiniBand), SRP (Mellanox InfiniBand), USB, vHost, etc.

4.2.3.1.2.1 Adding a new target (on the target host)

To add a new iSCSI target edit the /etc/tgt/targets.conf configuration file. This file contains many examples of different configuration options that have been commented out. A basic target may be defined as:

<target iqn.2008-09.com.example:server.target1>

backing-store /srv/images/iscsi-share.img direct-store /dev/sdd

</target>

This example defines a single target with two LUNs. LUNs are described with either the backing-store or direct-store directives where backing-store refers to either a file or a block device, and direct-store refers to local SCSI devices. Device parameters, such as serial numbers and vendor names, will be passed through to the new iSCSI LUN.

The target service is aptly named tgtd. To start it run:

# service tgtd start

To stop the tgtd service, run:

# service tgtd stop If there are open connections, use:

# service tgtd force-stop

Warning Using this command will terminate all target arrays.

4.2.3.1.3 WWID

Any connected SCSI device is assigned a unique device name. The device name begins with /dev/sd, for example /dev/sdd, /dev/sda. Additionally, each SCSI device has a unique World Wide Identifier (WWID). The identifier can be obtained from the device itself by issuing the inquiry command. On Linux systems you can find the WWIDs in the /dev/disk/by-id/

directory, in which you will find symbolic links whose names contain the WWID and which point to the assigned device name.

There are two types of WWIDs, known as ’page 0x83’ and ’page 0x80’ and any SCSI device has one of these.

For example, a device with a page 0x83 identifier would have:

scsi-3600508b400105e210000900000490000 -> ../../sda Or, a device with a page 0x80 identifier would have:

scsi-SSEAGATE_ST373453LW_3HW1RHM6 -> ../../sda

Red Hat Enterprise Linux automatically maintains the proper mapping from the WWID-based device name to a current /dev/sd name on that system. Applications can use the /dev/disk/by-id/ name to reference the data on the disk, even if the path to the device changes, and even when accessing the device from different systems. This also allows detection and access to plugable devices, say a photocamera. If it is plugged in, the system will sent a inquiry command and will create the proper symbolic link, hence allowing you to access the camera under the same name.

It is possible and feasible to have more than one path to a device, for example to offer a fail-over or fall-back option and/or to increase performance. Such ’paths’ might be implemented as additional network paths (preferably using their own network card) or over fibre (preferably using their own HBAs) etc. If there are multiple paths from a system to a device, a special kernel-module and associated daemons and other software will use the WWID to detect the paths. A single “pseudo-device”

in /dev/mapper/wwidwill be created by them, such as /dev/mapper/3600508b400105df70000e00000ac0000, which will give access to the device, regardless which path(s) to it there are in use, as long as at least one of them can be used (’is active’).

The command multipath -l shows the mapping to the non-persistent identifiers: Host:Channel:Target:LUN, /dev/sd name, and the major:minor number:

3600508b400105df70000e00000ac0000 dm-2 vendor,product [size=20G][features=1 queue_if_no_path][hwhandler=0][rw]

\_ round-robin 0 [prio=0][active]

\_ 5:0:1:1 sdc 8:32 [active][undef]

\_ 6:0:1:1 sdg 8:96 [active][undef]

\_ round-robin 0 [prio=0][enabled]

\_ 5:0:0:1 sdb 8:16 [active][undef]

\_ 6:0:0:1 sdf 8:80 [active][undef]

Device-mapper-multipath automatically maintains the proper mapping of each WWID-based device name to its corresponding /dev/sd name on the system. These names are persistent a cross path changes, and they are consistent when accessing the device from different systems. When the user_friendly_names feature (of device-mapper-multipath) is used, the WWID is mapped to a name of the form /dev/mapper/mpathn. By default, this mapping is maintained in the file /etc/multipath/

bindings. These mpath names are persistent as long as the file is maintained.

Note

Important If you use user_friendly_names, then additional steps are required to obtain consistent names in a cluster. Refer to the Consistent Multipath Device Names in a Cluster section in the Using DM Multipath Configuration and Administration book.

4.2.3.1.4 LUN

Another way to address a SCSI device is by using its LUN, the Logical unit number. The LUN is a three bit number (0..7) which indicates a device within a target. Any target should at least have a device with LUN 0, which can be used like any device, but is also capable of providing information about the other LUNs in the device (if any). If there is just one device in a target that device has LUN 0 as any target should have a LUN 0.

Say you bought a SCSI device that contained three disks (all connected over the same SCSI cable). To find out how many disks there are in your target (or tape units etc.) you would sent an inquiry to LUN 0. The device that has LUN 0 is designed to be able to tell you the configuration and might report that it knows of other devices. To address these the client uses the LUNs as specified by LUN 0.

Because SCSI disks were often refered to by their LUN, the habit stuck and nowadays the term is also used to refer to a logical disk in a SAN. But there can be many more LUNs in a SAN than the humble 7 that are used in older configurations. Also, such a SAN-LUN may well consist of series of disks, which might possibly be configured as a RAID, possibly subdivided in volumes etc. - to the client, it is just a disk with a LUN.

4.2.3.1.4.1 Configuring device name persistence (aka LUN persistence)

Linux device names may change upon boot. This can lead to confusion and damage to data. Therefore, there are a number of techniques that allow persistent names for devices. This section covers how to implement device name persistence in guests and on the host machine with and without multipath. Persistence simply means that you will have the same name for the device, regardles its access paths. So, your camera or disk is always recognised as such and will be issued the same device name, regardless where you plug it in.

4.2.3.1.4.2 Implementing device name persistence without multipath

If your system is not using multipath, you can use udev to implement LUN persistence. udev is a device manager that listens to an interface with the kernel. If a new device is plugged in or a device is being detached, udev will receive a signal from the kernel and, using a set of rules, will perform actions, for example assign an additional device name to the device.

Before implementing LUN persistence in your system you will first need to acquire the proper WWIDs from the devices you want to control. This can be done by using the scsi_id program. After you required the WWIDs of your devices, you will need to write a rule so udev can assign a persistent name to it.

A minor hurdle has to be taken first. The scsi_id program will not list any device by default and will only list devices it has been told to see (’whitelisted’ devices). To whitelist a device you will need to add its WWID to the scsi_id configuration file, which is /etc/scsi_id.conf. That’s safe, but a nuisance if you want to discover WWIDs of new devices. Hence you can specify the -g option to allow detection of any disk that has not been listed yet. If you feel that this should be the default behaviour, you may consider editing /etc/scsi_id.conf. If there is a line that says:

options=-b

delete it or comment it out. Then add a line that says:

options=-g

If this has been configured, next time a program uses the scsi_id command, it will add the -g option (and any other options you might have specified in the configuration file) and hence allow extracting the WWIDs from new devices. The -g should be used either by default from the configuration file or be set manually in any udev rule that uses the scsi_id command, or else your newly plugged in device won’t be recognised.

You then will need to find out what device file your new device was allocated by the kernel. This is routinely done by using the command dmesg after you plugged in the new device. Typically, you will see lines similar to these:

usb 1-5: new high speed USB device using ehci_hcd and address 25 usb 1-5: configuration #1 chosen from 1 choice

scsi7 : SCSI emulation for USB Mass Storage devices usb-storage: device found at 25

usb-storage: waiting for device to settle before scanning

Vendor: USB Model: Flash Disk Rev: 8.07

Type: Direct-Access ANSI SCSI revision: 04

SCSI device sdc: 7886848 512-byte hdwr sectors (4038 MB)

As you see, we plugged in a 4Gb Flash Disk that was assigned the device name ’sdc’. This device is now available as /sys/

block/sdcin the /sys pseudo-filesystem, and also as a devicefile /dev/sdc.

To determine the device WWID, use the scsi_id command:

# scsi_id -g -u -s /block/sdc 3200049454505080f

As you see, we used the -g forcibly. The -s option specifies that you want to use the device name relative to the root of the sysfs filesystem (/sys). Newer versions of scsi_id (e.g. those in RHEL6) do not allow use of sysfs names anymore, so should use the device name then:

/sbin/scsi_id -g -u -d /dev/sdc 3200049454505080f

The long string of characters in the output is the WWID. The WWID does not change when you add a new device to your system. Acquire the WWID for each device in order to create rules for the devices. To create new device rules, edit the 20-names.rulesfile in the /etc/udev/rules.d directory. The device naming rules follow this format:

KERNEL="sd<?>", BUS="scsi", PROGRAM="<scsi_id -options..>", RESULT="<WWID>", NAME="< ←-devicename>"

So, when you plug in a new device the kernel will signal udev it found a new SCSCI device. the PROGRAM will be executed and the RESULT should match. If so, the NAME will be used for the device file.

An example would be:

KERNEL="sd*", BUS="scsi", PROGRAM="/sbin/scsi_id -g -u -d /dev/$parent", RESULT

←-="3200049454505080f", NAME="bookone"

Or on RHEL6 systems:

KERNEL="sd*", BUS="scsi", PROGRAM="/sbin/scsi_id -g -u -s /block/$parent", RESULT

←-="3200049454505080f", NAME="bookone"

When the kernel allocates a new device which name matches the KERNEL pattern, the PROGRAM is executed and when the RESULT is found the NAME is created. Note the -g option that we have included so we can actually see the device, though it is not whitelisted.

The udevd daemon is typically started by the execution of a local startup script, normally done by the init process and initiated by a line in the /etc/inittab file. On older systems you would ensure that this line was in /etc/rc.local:

/sbin/start_udev

On newer systems, e.g. RHEL5 and RHEL6 the line should be in /etc/rc.sysinit.

4.2.3.1.4.3 Implementing device name persistence with multipath

You can implement device name persistence by using the multipath drivers and software. Of course, normally you would only use this if you have more than one path to your device. But you can actually define a multipath with only has one active path, and it will work just like any other disk. Just define an alias and the name will be persistent across boots.

The multipath software consists of a daemon, that guards the paths to the various devices. It executes the external command multipath on failure. The multipath command in turn will signal the daemon after it has reconfigured paths.

To implement LUN persistence in a multipath environment, you can define an alias name for the multipath device. Edit the device aliases in the configuration file of the multipath daemon, /etc/multipath.conf:

multipaths { multipath {

wwid 3600a0b80001327510000015427b625e alias backupdisk

} }

This defines a device that has a persistent device file /dev/mpath/backupdisk. That will grant access to the device -whatever its name - that has WWID 600a0b80001327510000015427b625e. The alias names are persistent across reboots.

In document The LPIC-2 Exam Prep. The LPIC-2 Exam Prep. Copyright 2013 Snow B.V. (Page 118-123)