Virtual machines can either be stored on hard disks local in the ESX(i) server, or on a network storage device such as a Storage Area Network (SAN) server. In both cases, the storage is added to the ESX(i) server and seen as datastores.
One advantage of having the virtual machines stored on a SAN server is that the virtual machines can be moved from one ESX(i) server to another. Since both ESX(i) servers have access to the same shared storage, it is only the execution context that needs to be moved rather than the much larger virtual machine hard disk.
This guide provides an overview of SAN and how it is used with VMware vSphere.
While SAN servers support either Fiber Channel, iSCSI, or both, this guide only provides information on iSCSI. Most major server vendors also supply SAN servers. The specific configuration of each SAN type is not documented here. Refer to the vendors specific guides for individual device configuration details.
It is essential that the configuration used is confirmed to function as expected before used for production. This includes using the performance monitors in the ESX(i) servers to ensure that load balancing is operational and to ensure that network redundancy is working by temporarily removing network connections.
SAN Hardware Section 12 Storage Area Network
SAN Hardware
SAN servers are highly redundant disk arrays that provide storage over a network.
Each SAN has redundant power supplies, redundant controller modules, redundant network connections per controller module and redundant disks.
The following parts as numbered in the Figure 33 are:
1. Power Supplies.
2. Fiber Channel host ports.
3. iSCSI ports (host or replication).
4. CLI port (mini-USB).
5. Reserved for future use.
6. Management Ethernet port.
7. SAS expansion port.
Figure 33. Rear View of an SAN Device
Section 12 Storage Area Network SAN Storage Configuration
SAN Storage Configuration
SAN servers typically consists of 12-24 hard disks. These are normally configured as one or more RAID 5 (block-level striping with distributed parity) disk groups with a number of disks set as hot spares or hot standby used as a failover
mechanism to provide reliability in System Configurations. The SAN server vendor may have best practices for allocation of the storage.
When combining disks to create RAID arrays, a storage item is created which is used to create virtual disks. These are the entities that become available as datastores in ESX(i).
When formatting datastores, the file block size may need to be changed to accommodate the largest file that is created in the datastore.
Always remember that the largest size for a datastore is 2TB - 512Bytes.
Figure 34. SAN Server Storage Configuration
SAN Networking Section 12 Storage Area Network
SAN Networking
Figure 35 and Figure 36 provides an overview of the networking required for a SAN server implantation.
Figure 35. SAN Storage iSCSI Network
Figure 36. SAN Storage Management Network
Section 12 Storage Area Network SAN Networking
One or more ESX(i) servers are connected to multiple network switches through multiple network connections. The SAN server is also connected to the switches through multiple network connections.
There are two types of network traffic in this configuration. One is for access to the shared storage through iSCSI. The other is for configuration and management of the servers. To provide security and reliability, the two types of traffic are separated using a VLAN.
To reduce single point of failures, separate network adapters should be used in the ESX(i) servers for the storage network as shown in the Figure 37.
While the multiple network connections provide redundancy, it also provides load balancing. Figure 38 provides an example of IP addresses for the storage
network.
When configuring the IP addresses in the SAN server, the VLAN settings may be in the advanced settings as shown in the Figure 39.
In the Figure 40, it is observed that the SAN server does not support VLAN. In this case, separate network switches for iSCSI and management traffic should be used.
Figure 37. Multiple Network Adapter Connection for Higher Reliability
SAN Networking Section 12 Storage Area Network
Figure 38. IP Address for the Storage Network
Section 12 Storage Area Network SAN Networking
Figure 39. Setting the VLAN ID for the iSCSI Ports
SAN Networking Section 12 Storage Area Network
Figure 40. SAN Server iSCSI IP Configuration
Section 12 Storage Area Network ESX Server Network Configuration
ESX Server Network Configuration
Implementation of networking for the virtual solution involves separating
functionality and implementing redundancy. There should be separate networks for the management and SAN iSCSI storage networks. This prevents management network traffic from interfering with the storage network traffic. Figure 41 shows the network configuration. Hardware vendor guidelines may recommend different configurations but, the configuration should always be verified that network utilization is optimized.
Adding in Datastores
This description is based on using the iSCSI Software Adapter in the ESX Server.
Perform the following steps in the ESX Server:
1. Select the Configuration > Storage Adapters Configuration.
2. Select iSCSI Software Adapter.
3. On the Details, select Properties....
4. In the iSCSI Initiator Properties, select Configure....
5. Select Enabled and click OK.
Figure 41. ESX Server Network Configuration
Adding in Datastores Section 12 Storage Area Network
6. On the Dynamic Discovery Tab, for each SAN IP Addresses, perform the following steps.
– Select Add.... Enter the IP Address of the iSCSI Server and click OK.
– Click Close once all the IP Addresses are added.
– A message appears to Rescan the Host Bus Adapter, click Yes to rescan.
Load Balancing
Perform the following steps to manage the paths:
1. Select the iSCSI Software Adapter.
2. On the Details, the devices list appears as shown in the Figure 42.
If the iSCSI storage server is not used before, the datastores must be added and formatted. The procedure is same as adding a local storage disk.
I
When formatting the datastores it is important to consider the largest file size and set an appropriate block size.
If the iSCSI storage server is already been used in an ESX server, then the datastores appears in the storage configuration without the intervention of the user.
Figure 42. iSCSI Device Details
Section 12 Storage Area Network Adding in Datastores
4. On the Manage Paths window, select Round Robin from the Path Selection (Figure 43).
Performance
It is essential that the configuration provides the best performance before using in a production environment. This involves monitoring the throughput to the datastores and ensuring that all networks are being used.
Figure 43. iSCSI Path Selection Policy
Adding in Datastores Section 12 Storage Area Network
In the network performance chart below, the 4 network adapters that are supporting the SAN server 1,3,4,7 (Figure 44) are all providing throughput to the SAN sever.
If the configuration is not correctly setup, not all the network connections used for communication with the SAN server. It displays the network traffic. It is important to check the average/maximum values as there may be instances where the network traffic was sampled when there was no traffic.
The following are the areas that must be checked:
• Are the network switches in the ESX(i) server configured correctly.
• Are the VLANs setup correctly in the ESX(i) server and the SAN server.
• Can the SAN server be pinged on each of the IP addresses from the ESX(i) server. Note that vmkping is used from the console of the ESX(i) server.
Figure 44. Load balancing of SAN Servers
Section 12 Storage Area Network Adding in Datastores
• Does the SAN server has a preferred path that needs to be reset or reconfigured.
• Contact the SAN supplier for additional information regarding configuration.
Adding in Datastores Section 12 Storage Area Network