Tracing the I/O path for network storage

There is a fundamental difference in the way that client/server I/O requests are driven over a TCP/IP LAN compared to I/O requests to directly attached or SAN attached storage.

Device specific requests to local or SAN attached device

Block I/O / data / storage location

Tracing a Local or FC SAN Block I/O

Data in blocks to sector location on disk

Storage I/O Bus Adapter

File I/O requests: 1. Raw I/O 2. File system I/O

Operating System

Device Driver

Host Bus Adapter

Application Server Volume Manager Raw Partition Manager Database Application File System Database System 1 2 1 2

2.8.1 Redirected I/O

Software on the client presents a logical view of the data files on the remote appliance, as though they are stored on the client’s own local disk. This

“virtualization” of the storage provides transparent access to the remote storage. However, we have seen that the application normally relies on file systems, volume managers, and device drivers to convey all the required information about the I/O operation to the storage device. (See 2.7, “Tracing the I/O path for local storage” on page 98.)

With a network I/O, much of this device-specific information is lacking because the storage and its file system is remote from the application that requests the I/O. To overcome this, the client software must “redirect” the I/O over the network. An

I/O redirector

is located in the client I/O path in front of the client’s own file system. It presents a common view of the client’s own local file system, and, transparently, the remote server’s file system. The I/O redirector has no knowledge of the metadata relating to either of the files systems. But the local and remote storage devices may, in reality, have quite different characteristics. On receipt of the I/O request to a file that is located in the remote NAS appliance, the following occurs:

򐂰 The I/O redirector performs what is called a “mapped drive” in the Windows world, or a “remote mount” in UNIX.

򐂰 The I/O request is directed away from the local I/O path to an alternative path over the network, which accesses the remote file server.

Since the client system has no awareness of the device characteristics on which the data is stored on the remote server, all redirected I/Os must be done at the file (byte range) level. This is termed a “file I/O.”

The client is attached to the LAN by a Network Interface Card (NIC). Since the NIC uses a network protocol, such as the TCP/IP stack, the I/O operation must be transferred using a network protocol. Now one of the network file protocols (such as NFS or CIFS) comes into play as a kind of network device driver. In effect, the network file protocol lies on top of the lower level communications protocol stack, such as TCP/IP. It is the TCP/IP protocol that carries the redirected I/O through the NIC onto the network. On a LAN the media access control layer used is typically the Ethernet CSMA/CD protocol. (See 2.3.3, “The CSMA/CD protocol” on page 73.)

When the remote server, or NAS appliance, receives the redirected I/O, the requests are “unbundled” from their TCP/IP network protocols. This is done by specialized device drivers in the receiving NIC. They are then sent to the receiving network file handling protocol. This controls tracking information in order to be able to exchange information with the client initiator. Now the request is submitted to the NAS appliance’s operating system, which manages the scheduling of the I/O, and security processes to the local disk. From then on the I/O is handled more or less like a local I/O. It is routed via the appliance’s file system, which establishes the file’s identity and directory, and converts the request to a storage system protocol (that is, a block I/O operation). Finally, the I/O request is handed to the volume manager for mapping to the device, and then on to the device driver and the I/O bus (or Fibre Channel SAN HBA attachment).

The receiving NAS device must keep track of the initiating client’s details so that the response can be directed back to the correct network address. The route for the returning I/O follows more or less the reverse path outlined above.

2.8.2 Network File I/O summary

One of the key differences of a NAS device, compared to DAS or other network storage solutions such as SAN or iSCSI, is that all I/O operations use file-level I/O protocols. In summary, the network access methods—NFS, CIFS, and Novell Netware—can only handle File I/O requests to the remote file system located in the operating system of the NAS device. This is because they have no

knowledge of the characteristics of the remote storage device. I/O requests are packaged by the initiator into the TCP/IP protocol stack, in order to move across the IP network. The remote NAS OS file system converts the request to block I/O and reads or writes the data to the NAS disk storage. This is illustrated in Figure 2-9 on page 104. (It is clear from this diagram that the network file I/O process involves many more steps than storage protocol (block) I/O that was illustrated in Figure 2-8.) This software stack overhead is a factor in comparing performance of a NAS I/O to a DAS or SAN-attached I/O.

It is important to note that a database application accessing a remote file located on a NAS device, by default, is configured to run with

file system I/O

. It cannot use

raw I/O

to achieve improved performance.

Figure 2-9 Tracing a network file I/O’

In document IP Storage Networking: (Page 119-122)