The Host Server. Linux Configuration Guide. July The Data Infrastructure Software Company

(1)

The Host Server

Linux Configuration Guide

(2)

Overview

This document provides Linux-specific settings when serving Virtual Disks from a DataCore Server.

Fundamental Linux administration skills are assumed; including how to connect Linux Hosts to storage array target ports (i.e. DataCore Server Front End ports) via either Fibre Channel or iSCSI, along with the process of discovering, mounting and formatting disk devices in general.

Recent changes made to this document

New information added since last update (June 2016)

Updated

This document has been reviewed for SANsymphony 10.0 PSP 5. No updates were required.

Added April 2016

Linux compatibility lists

Red Hat Enterprise Linux Versions 6.7 – 6.8

Previous changes made to this document

(4)

Linux compatibility lists

RedHat Enterprise Linux

SANsymphony

9.0 PSP 4 Update 4 (1) 10.0 (all versions)

Version With ALUA Without ALUA With ALUA Without ALUA

5.4 and earlier Not Supported Not Supported Not Supported Not Supported

5.5 – 5.10 Not Supported Not Supported Not Supported Not Supported

6.0 – 6.3 Supported Supported Not Qualified Not Qualified

6.4 Supported Supported Not Qualified Not Qualified

6.5 – 6.6 Not Qualified Not Qualified Supported Not Qualified

6.7 – 6.8 Not Supported Not Supported Not Qualified Not Qualified

7.0 Not Supported Not Supported Supported Not Qualified

7.1 – 7.2 Not Supported Not Supported Not Qualified Not Qualified

Compatibility notes

Please see page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'.

Fibre Channel and iSCSI Connections

DataCore Software supports RedHat Linux Hosts using either Fibre Channel or iSCSI Connections to SANsymphony Front-End (FE) Ports for any Virtual Disk type.

SCSI UNMAP

RedHat Linux Hosts are supported using SCSI UNMAP commands - see 'Appendix C - Reclaiming Storage' on page 24.

1

SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now ‘End of Life’. Please see: End of Life Notifications http://datacore.custhelp.com/app/answers/detail/a_id/1329

(5)

Linux compatibility lists

SUSE Linux Enterprise Server

SANsymphony

10.x and

earlier Not Supported Not Supported Not Supported Not Supported

11.0 (no SP) Not Supported Supported Not Supported Not Supported

11.0 SP 1 Not Supported Not Supported Not Supported Not Supported

11.0 SP 2 Supported Supported Not Qualified Not Qualified

11.0 SP 3 Not Qualified Not Qualified Supported Not Qualified

11.0 SP 4 Not Supported Not Supported Not Qualified Not Qualified

12.x Not Supported Not Supported Not Qualified Not Qualified

Compatibility notes

Fibre Channel and iSCSI Connections

SCSI UNMAP

SUSE Linux Hosts are not supported using SCSI UNMAP commands.

(6)

Linux compatibility lists

Ubuntu

SANsymphony

13.x and

earlier Not Supported Not Supported Not Supported Not Supported

14.04 LTS Not Supported Not Qualified Not Supported Supported

14.10 Not Supported Not Qualified Not Supported Not Supported

15.x Not Supported Not Qualified Not Supported Not Qualified

16.x Not Supported Not Supported Not Qualified Not Qualified

Note: This table applies only to the Linux distribution provided by Canonical Ltd and does not include any Debian Linux distribution (or other distributions that are themselves either based on Debian or Ubuntu’s own codebases).

Compatibility notes

Fibre Channel and iSCSI connections

Note: If the Host is running SAP HANA then only Fibre Channel connections are supported. Please see page 16 for more information.

SCSI UNMAP

Ubuntu Linux Hosts have not been tested using SCSI UNMAP commands – also see 'Appendix C - Reclaiming Storage' on page 25.

1

Please see the compatibility notes on page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'.

(7)

Linux compatibility lists

Compatibility notes

Supported

These Linux/SANsymphony combinations have been tested using all the host-specific settings listed in this document against all Virtual Disk types. Mirrored and Dual Virtual Disks have been tested for 'high availability' in all possible failure scenarios.

Not Qualified

These Linux/SANsymphony combinations have not been tested against any Mirrored or Dual Virtual Disks types. DataCore therefore cannot guarantee 'high availability' in any failure scenario (even if all host-specific settings listed in this document are followed) however, self-qualification may be possible. For more information on this please see:

http://datacore.custhelp.com/app/answers/detail/a_id/1506

Support for any Linux versions that are considered ‘End of Life’ by the vendor and are listed as 'Not Qualified' can still be self-qualified but only if there is an agreed ‘support contract’ with your Linux vendor. In this case though, DataCore Technical Support will not do root-cause analysis for SANsymphony in the case of any future issues but will offer 'best effort' support to get Hosts accessing any SANsymphony Virtual Disks.

Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Qualified' combinations of Linux/SANsymphony.

Not Supported

These Linux/SANsymphony combinations have usually failed one or more of our 'high

availability' tests when using Mirrored or Dual Virtual Disks types; but also may simply be where an Operating System's own requirements (or limitations) due to its age make it impractical to test. Entries marked as 'Not Supported' can never be self-qualified. Mirrored or Dual Virtual Disks types are configured at the end-user's own risk.

Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Supported' combinations of Linux/SANsymphony.

End of Life Linux distributions

Support for any Linux distribution that is considered ‘End of Life’ by the vendor or has no active development/Long Term Support can still be self-qualified but only if there is an agreed

‘support contract’ with the supplier/vendor or the Linux Distribution.

In this case, DataCore Technical Support will help the customer to get the Host Operating system accessing Virtual Disks, but will not then do any root-cause analysis.

(8)

Other Linux distributions

All other Linux distributions that are not listed in this document are considered as unqualified for mirrored or dual Virtual Disks when using SANsymphony and, just like all the unqualified versions of Linux that are listed in this document, the behavior with SANsymphony cannot be guaranteed.

These other distributions can also ‘self-qualified’ by following the same process mentioned in the compatibility notes on page 7.

Self-qualifying 'other' Linux distributions

DataCore recommends using the same configuration settings as those listed for Ubuntu (i.e. the less complex, non-ALUA settings) for self-qualifying. Then assuming this passes all the tests, either of the ALUA-based configuration settings can then be tested.

Note: Non-mirrored Virtual Disks are always considered as supported even for 'other' Linux distributions used with SANsymphony.

(9)

The DataCore Server's settings

These are the Host-specific settings that need to be configured directly on the DataCore Server.

Operating system type

See the Registering Hosts section from the SANsymphony Help: http://www.datacore.com/SSV-Webhelp/Hosts.htm

RedHat Enterprise Linux

When registering the Host choose the 'Linux (all other distributions)' menu option.

SUSE Linux Enterprise Server

When registering the Host choose the 'Linux SUSE Enterprise Server 11 ' menu option.

Ubuntu Linux

When registering the Host choose the 'Linux (all other distributions)' menu option.

Port roles

Ports used for serving Virtual Disks to Hosts should only have the Front End (FE) role enabled. Mixing other Port Role types may cause unexpected results as Ports that only have the FE role enabled will be turned off when the DataCore Server software is stopped (even if the physical server remains running). This helps to guarantee that any Hosts do not still try to access FE Ports, for any reason, once the DataCore Software is stopped but where the DataCore Server remains running. Any Port with the Mirror and/or Back End role enabled do not shut off when the DataCore Server software is stopped but still remain active.

Multipathing support

The Multipathing Support option should be enabled so that Mirrored Virtual Disks or Dual Virtual Disks can be served to Hosts from all available DataCore FE ports. Also see the

Multipathing Supportsection from the SANsymphony Help: http://www.datacore.com/SSV-Webhelp/Hosts.htm

Non-mirrored Virtual Disks and Multipathing

Non-mirrored Virtual Disks can still be served to multiple Hosts and/or multiple Host Ports from one or more DataCore Server FE Ports if required; in this case the Host can use its own

multipathing software to manage the multiple Host paths to the Single Virtual Disk as if it was a Mirrored or Dual Virtual Disk.

Note: Hosts that have non-mirrored Virtual Disks served to them do not need Multipathing Support enabled unless they have other Mirrored or Dual Virtual Disks served as well.

(10)

The DataCore Server's settings

Asymmetrical Logical Unit Access (ALUA) support

The ALUA support option should be enabled if required and if Multipathing Support has been also been enabled (see above). Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA. More information on 'Preferred Servers' and 'Preferred Paths' used by the ALUA function can be found on in Appendix A on page 21.

Serving Virtual Disks to the Hosts for the first time

DataCore recommends that before serving Virtual Disks for the first time to a Host, that all DataCore Front-End ports on all DataCore Servers are correctly discovered by the Host first. Then, from within the SANsymphony Console, the Virtual Disk is marked Online, up to date and that the storage sources have a host access status of Read/Write.

Virtual Disks LUNs and serving to more than one Host or Port

DataCore Virtual Disks always have their own unique Network Address Authority (NAA)

identifier that a Host can use to manage the same Virtual Disk being served to multiple Ports on the same Host Server and the same Virtual Disk being served to multiple Hosts.

See the SCSI Standard Inquiry Datasection from the online Help for more information on this: http://www.datacore.com/SSV-Webhelp/Changing_Virtual_Disk_Settings.htm

While DataCore cannot guarantee that a disk device's NAA is used by a Host's operating system to identify a disk device served to it over different paths generally we have found that it is. And while there is sometimes a convention that all paths by the same disk device should always using the same LUN 'number' to guarantees consistency for device identification, this may not be technically true. Always refer to the Host Operating System vendor’s own documentation for advice on this.

DataCore's Software does, however always try to create mappings between the Host's ports and the DataCore Server's Front-end (FE) ports for a Virtual Disk using the same LUN number(1) where it can. The software will first find the next available (lowest) LUN 'number' for the Host-DataCore FE mapping combination being applied and will then try to apply that same LUN number for all other mappings that are being attempted when the Virtual Disk is being served. If any Host-DataCore FE port combination being requested at that moment is already using that same LUN number (e.g. if a Host has other Virtual Disks served to it from previous) then the software will find the next available LUN number and apply that to those specific Host-DataCore FE mappings only.

1

The software will also try to match a LUN 'number' for all DataCore Server Mirror Port mappings of a Virtual Disk too, although the Host does not 'see' these mirror mappings and so this does not technically need to be the same as the Front End port mappings (or indeed as other Mirror Path mappings for the same Virtual Disk). Having Mirror mappings using different LUNs has no functional impact on the Host or DataCore Server at all.

(11)

The Linux Host's settings

Operating system settings

Disk Timeouts

Set the SCSI Disk timeout value to 80 seconds for any DataCore Virtual Disk devices.

echo 80 > /sys/block/[disk_device]/device/timeout

For example if the two DataCore Virtual Disks are using /dev/sda and /dev/sdb respectively: echo 80 > /sys/block/sda/device/timeout

echo 80 > /sys/block/sdb/device/timeout

This will change the SCSI Disk timeout value to 80 seconds for those particular disk devices. The SCSI Disk timeout can then be verified by running the cat command on the disk device directly

cat /sys/block/sda/device/timeout

Note: The SCSI Disk timeout may revert to the system default after a reboot. Please contact your Linux vendor or consult their documentation for details on creating a ‘rules’ file in /etc/udev/ to help resolve this behavior.

Fibre Channel device timeouts

Do not specify any fibre channel-specific device (i.e. lpfc_devloss_tmo for Emulex or,

qlport_down_retry for the QLogic) but use the values in the multipath configuration file (see the next page).

(12)

The Linux Host's settings

Multipath configuration settings

Polling Interval

In the defaults section of the multipath.conf file, the polling_interval must be set to 60 defaults {

polling_interval 60

}

This is a DataCore-required value which helps prevent excessive Host attempts to check for a Virtual Disk Storage Source’s Host Access value, after a failure, is set as Offline. Smaller interval settings will interfere with overall Host performance.

Do not add this parameter to the 'device' section (discussed on the next page) by mistake; else the setting will not work as expected.

Blacklist exceptions

Usually all storage vendor devices are specified under a separate device sub-section within the

blacklist_exceptions section. blacklist_exceptions {

vendor "DataCore"

product "Virtual Disk"

(13)

The Linux Host's settings

The 'device' section

ALUA-enabled Hosts

Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA

device {

vendor "DataCore"

path_checker tur prio alua failback 10 no_path_retry fail dev_loss_tmo infinity fast_io_fail_tmo 5 rr_min_io_rq 100

# Alternative option – See notes below

# rr_min_io 100

path_grouping_policy group_by_prio

# Alternative policy - See notes below

# path_grouping_policy failover

# optional - See notes below

# user_friendly_names yes

}

Without ALUA enabled

Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA

device {

vendor "DataCore"

path_checker tur

failback 10

dev_loss_tmo infinity

fast_io_fail_tmo 5

no_path_retry fail

# optional - See notes below

# user_friendly_names yes

(14)

The Linux Host's settings

Device section notes

Note: All entries listed are required by SANsymphony and are case sensitive.

dev_loss_tmo infinity

Also requires the ‘fast_io_fail_tmo 5’ setting (see next). The dev_loss_tmo setting controls the length of time (normally indicated in seconds) before a PATH to a Virtual Disk, that has since become unavailable to the Linux Host, is removed from the Linux operating system. For

example; when a DataCore Server is stopped or a PATH to a DataCore Server’s Virtual Disk is set to ‘Host Access: Not Allowed’.

Once a PATH to a Virtual Disk has been removed by the Linux operating system it can then only usually be re-established by ‘manual intervention’ (e.g. user-initiated rescan on the Linux Host). The ‘infinity’ value prevents the PATH from being removed. If the ‘fast_io_fail_tmo 5’ setting is not present in the multipath.conf file, the ‘infinity’ setting is ignored and the dev_loss_tmo

value defaults to ‘600’ (10 minutes).

Note: Some older kernels of RedHat Enterprise Linux and SUSE Linux Enterprise Server 11 SP2 and earlier do not support the ‘infinity’ value and it will be ignored (and may also post an error in syslog). In that case, a default value - usually ‘600’ seconds - will be applied instead.

Use the ‘cat’ command to verify that any DataCore Virtual Disks detected by the Linux Host are using the ‘infinity’ value correctly.

A simple example:

sleshost3:~ # cat /sys/class/fc_remote_ports/rport-*\:*-*/dev_loss_tmo

30 30 2147483647 2147483647 2147483647 30

Note: The value of ‘2147483647’ indicates a dev using the infinity setting.

fast_io_fail_tmo

This is required by the ‘dev_loss_tmo infinity’ setting (see previous). Do not use any other value other than ‘5’, otherwise the dev_loss_tmo setting will use a larger, default value (usually ‘600’ seconds).

failback

This adds an extra ‘wait’ period (10 seconds) that helps to prevent unnecessary ‘failback’ attempts to a Virtual Disk whose Storage Source’s Host Access value is still set as Not Allowed.

(15)

The Linux Host's settings

no_path_retry

Required for any Linux Hosts configured for ALUA and want to use the Preferred Server setting set to ‘ALL’. See Appendix A on page 21 about information regarding the ‘ALL’ setting.

path_checker

This is a DataCore-required value. No other value should be used.

path_grouping_policy group_by_prio or path_grouping_policy group_by_failover

One of two possible values is required – ‘group_by_prio’ or ‘failover’ - to avoid any other, non-DataCore, setting in the multipath.conf (i.e. from other storage arrays attached to the Linux Host) from taking precedence.

Note: The ‘failover’ value is unqualified for RHEL 6.5 and greater.

In either case, make sure the Preferred Server setting on the DataCore Server is either set to 'Auto Select' or an explicit Server Name. See Appendix A on page 21 about information regarding the 'Auto Select' setting.

rr_min_io_rq

This is a DataCore-required value for Linux Hosts that are running kernels newer than 2.6.30. Older versions must use the option ‘rr_min_io 100’ instead (see next).

rr_min_io

This is a DataCore-required value for Linux Hosts that are running kernels older than 2.6.31. Newer versions must use the option ‘rr_min_io_rq 100’ instead (see previous).

user_friendly_names

This is an optional setting and simply specifies that the operating system should use the

‘/etc/multipath/bindings’ file to assign a persistent and unique alias to the multipath device, in the form of mpathn.

If this value is set to 'no' (or omitted completely) the operating system will use a WWID as an alias for the multipath device instead.

(16)

Linux applications

SAP HANA

Note: For more detailed information about HANA operation and configuration always consult the HANA configuration documents available from SAP.

Sizing guidelines

Please refer to the documents:

SANsymphony with SAP HANA - Sizing Guidelines

http://datacore.custhelp.com/app/answers/detail/a_id/1619

SAP HANA versions

Versions SPS09 and SPS10 have both been certified.

DataCore Software version requirements

SANsymphony 10.0 PSP 3 or greater using Fibre Channel connections for Front-end, Mirror and Back-end connections. ISCSI connections are not certified and should not be used.

Host operating system requirements

SUSE Linux Enterprise Server 11.0 Service Pack 3 with the additional updates (or later) installed:

kpartx-0.4.9-0.105.1.8637.2.PTF.933282.x86_64.rpm

multipath-tools-0.4.9-0.105.1.8637.2.PTF.933282.x86_64.rpm

Host settings requirements

The Linux Host must be configured for ALUA. Hosts without ALUA are not certified.

Mount point and filesystem requirements

/hana/log Use XFS

/hana/data Use XFS

/hana/shared Use OCFS2

Additional file system settings

/hana/log: None

/hana/data: async_write_submit_active = on

async_write_submit_blocks = all

/hana/shared: None

(17)

Linux Applications – SAP HANA

su <SID>adm

cd /usr/sap/<SID>/HDB<Instance> source hdbenv.sh

hdbparam --paramset "fileio[DATA].async_write_submit_active=on" hdbparam --paramset "fileio[DATA].async_write_submit_blocks=all" exit

The global.ini setting

Use the ‘PRES Type 5’ by adding the following line to the ‘storage’ section

(18)

Linux Applications – SAP HANA

An example of a HANA 3-node configuration

In this example, two of the HANA nodes are ‘active’ and the third node is ‘standby’. Active Server – host1

/hana/log/DB1/mnt00001 <- Mounts host1 Log_vDisk

/hana/data/DB1/mnt00001 <- Mounts host1 Data_vDisk

/hana/log/DB1/mnt00002 <- Mounts host2 Log_vDisk

/hana/data/DB1/mnt00002 <- Mounts host2 Data_vDisk

/hana/shared <- Mounts shared OCFS2 filesystem vDisk

Active Server – host2

/hana/log/DB1/mnt00002 <- Mounts host2 Log_vDisk

/hana/data/DB1/mnt00002 <- Mounts host2 Data_vDisk

Standby Server – host3 (in failover host role)

Notes: Filesystem entries hi-lighted in red are those that are mounted on that particular host when the database is started. Those that are not hi-lighted in red are still available to the host but are not mounted.

In the case of a ‘failover event’ the HANA application will determine which of the other hosts will then mount and use the failed host’s file systems. Because any host can, potentially, require access to any of the other hosts’ file systems all SANsymphony Virtual Disks used for the SAP HANA data or log filesystems must be served to all SAP HANA hosts – active and standby.

(19)

Known issues

The following is intended to make DataCore Software customers aware of any issues that may affect performance, access or generally give unexpected results under certain conditions when Linux Hosts are used with SANsymphony.

Some of the issues here have been found during DataCore’s own testing but many others are issues reported by DataCore Software customers, where a specific problem had been identified and then subsequently resolved.

DataCore cannot be held responsible for incorrect information regarding Linux products. No assumption should be made that DataCore has direct communication with any of the Linux vendors regarding the issues listed here and we always recommend that users contact their own Linux vendor directly to see if there are any updates or fixes since they were reported to us.

For ‘Known issues’ for DataCore’s own Software products, please refer to the relevant DataCore Software Component’s release notes.

Formatting a Virtual Disk may take longer than expected

Since SANsymphony-V 9.0 PSP4, Linux Hosts have been able to take advantage of

SANsymphony’s SCSI-UNMAP support - See the Appendix C - ‘Reclaiming Storage' on page 24. However for RHEL and SLES Hosts, this will result in the mkfs command sending additional ‘discard’ commands during the format process, resulting in longer format times. Ubuntu has not been tested with SCSI UNMAP so is considered unqualified.

Use the ‘-K’ option while formatting to disable the ‘discard’ command during formatting. Also see: http://linux.die.net/man/8/mkfs.xfs and http://linux.die.net/man/8/mkfs.ext4

Refer to your own man page to be sure your installation supports this option.

Ext3 filesystems use excessive storage allocations

The Ext3 filesystem will use significant amounts of SAUs during the ‘Writing superblocks and filesystem accounting information’ phase of the filesystem creation.

Care must be taken so as not to completely use all the SAUs from the Disk Pool; and if Ext3 is required, then use a small (i.e. 4MB) SAU size. Other filesystem types do not seem to exhibit this behavior and use only a few SAUs during filesystem creation.

(20)

Known issues

Manual rescans are needed to update previously failed paths for Ubuntu Hosts

DataCore have not been able to get Ubuntu to automatically re-detect paths to mirrored Virtual Disks that have failed or have been removed (e.g. after stopping a DataCore Server) and are then subsequently made available again. Manual intervention is required.

Use the command

multipath -ll

Establish which paths are currently failed and then use one of the commands

rescan-scsi-bus

From the ‘scsitools’ package – usually installed by

apt-get install scsitools

or run the command

echo 1 > /sys/block/[disk_device_name]/device/rescan

For example

echo 1 > /sys/block/sdb/device/rescan

will send a signal to the disk device and ‘force’ the operating system to update the path’s status properly.

(21)

Appendix A

Preferred Server & Preferred Path settings

See the Preferred Servers and Preferred Pathssections from the SANsymphony Help: http://www.datacore.com/SSV-Webhelp/Port_Connections_and_Paths.htm

Without ALUA enabled

If Hosts are registered without ALUA support, the Preferred Server and Preferred Path settings will serve no function. All DataCore Servers and their respective Front End (FE) paths are considered ‘equal’.

It is up to the Host’s own Operating System or Failover Software to determine which DataCore Server is its preferred server.

With ALUA enabled

Setting the Preferred Server to ‘Auto’ (or an explicit DataCore Server), determines the DataCore Server that is designated ‘Active Optimized’ for Host IO. The other DataCore Server is

designated ‘Active Non-Optimized’.

If for any reason the Storage Source on the preferred DataCore Server becomes unavailable, and the Host Access for the Virtual Disk is set to Offline or Disabled, then the other DataCore Server will be designated the ‘Active Optimized’ side. The Host will be notified by both DataCore Servers that there has been an ALUA state change, forcing the Host to re-check the ALUA state of both DataCore Servers and act accordingly.

If the Storage Source on the preferred DataCore Server becomes unavailable but the Host Access for the Virtual Disk remains Read/Write, for example if only the Storage behind the DataCore Server is unavailable but the FE and MR paths are all connected or if the Host physically becomes disconnected from the preferred DataCore Server (e.g. Fibre Channel or iSCSI Cable failure) then the ALUA state will not change for the remaining, ‘Active

Non-optimized’ side. However, in this case, the DataCore Server will not prevent access to the Host nor will it change the way READ or WRITE IO is handled compared to the ‘Active Optimized’ side, but the Host will still register this DataCore Server’s Paths as ‘Active Non-Optimized’ which may (or may not) affect how the Host behaves generally.

(22)

Appendix A - Preferred Server & Preferred Path settings

In the case where the Preferred Server is set to ‘All’, then both DataCore Servers are designated ‘Active Optimized’ for Host IO.

All IO requests from a Host will use all Paths to all DataCore Servers equally, regardless of the distance that the IO has to travel to the DataCore Server. For this reason, the ‘All’ setting is not normally recommended. If a Host has to send a WRITE IO to a ‘remote’ DataCore Server (where the IO Path is significantly distant compared to the other ‘local’ DataCore Server), then the WAIT times accrued by having to send the IO not only across the SAN to the remote DataCore Server, but for the remote DataCore Server to mirror back to the local DataCore Server and then for the mirror write to be acknowledged from the local DataCore Server to the remote DataCore Server and finally for the acknowledgement to be sent to the Host back across the SAN, can be significant.

The benefits of being able to use all Paths to all DataCore Servers for all Virtual Disks are not always clear cut. Testing is advised.

For Preferred Path settings it is stated in the SANsymphony Help:

A preferred front-end path setting can also be set manually for a particular virtual disk. In this case, the manual setting for a virtual disk overrides the preferred path created by the preferred server setting for the host.

So for example, if the Preferred Server is designated as DataCore Server A and the Preferred Paths are designated as DataCore Server B, then DataCore Server B will be the ‘Active Optimized’ Side not DataCore Server A.

In a two-node Server group there is usually nothing to be gained by making the Preferred Path setting different to the Preferred Server setting and it may also cause confusion when trying to diagnose path problems, or when redesigning your DataCore SAN with regard to Host IO Paths. Where there are three or more Servers in a Server Group, and where one or more of these DataCore Servers shares Mirror Paths between different DataCore Servers then setting the Preferred Path makes more sense. So for example, DataCore Server A has two mirrored Virtual Disks, one with DataCore Server B, and one with DataCore Server C and DataCore Server B also has a mirrored Virtual Disk with DataCore Server C then using just the Preferred Server setting to designate the ‘Active Optimized’ side for the Host’s Virtual Disks becomes more complicated. In this case the Preferred Path setting can be used to override the Preferred Server setting for a much more granular level of control.

(23)

Appendix B

Configuring Disk Pools

See Creating Disk Pools and Adding Physical Disksfrom the SANsymphony Help: http://www.datacore.com/SSV-Webhelp/About_Disk_Pools.htm

The smaller the SAU size, the larger the number of indexes are required, by the Disk Pool driver, to keep track of the equivalent amount of allocated storage compared to a Disk Pool with a larger SAU size; e.g. there are potentially four times as many indexes required in a Disk Pool using a 32MB SAU size compared to one using 128MB – the default SAU size.

As SAUs are allocated for the very first time, the Disk Pool needs to update these indexes and this may cause a slight delay for IO completion and might be noticeable on the Host. However this will depend on a number of factors such as the speed of the physical disks, the number of Hosts accessing the Disk Pool and their IO READ/WRITE patterns, the number of Virtual Disks in the Disk Pool and their corresponding Storage Profiles.

Therefore, DataCore usually recommend using the default SAU size (128MB) as it is a good compromise between physical storage allocation and IO overhead during the initial SAU allocation index update. Should a smaller SAU size be preferred, the configuration should be tested to make sure that a potential increased number of initial SAU allocations does not impact the overall Host performance.

(24)

Appendix C

Reclaiming storage

Using SCSI UNMAP commands

Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support SCSI UNMAP commands from the Host. As of SANsymphony-V 9.0 PSP4 there is support for SCSI UNMAP commands which, when used in conjunction the Linux fstrim command or the ‘mount –o discard’ option (for some Linux file systems), allows Hosts to trigger the Automatic Reclamation function on served Virtual Disks. See the following online resources:

The Linux man-pages project

http://man7.org/linux/man-pages/man8/fstrim.8.html

From RedHat’s Storage Administration Guide: Section: ‘2.5. Discard unused blocks’

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Storage_Administration_Guide/

Always refer to your Linux Vendor to determine which file systems are supported for either the

fstrim command and/or the ‘mount –o discard’ option for the version of RHEL or SLES that you currently have installed on the Host.

It is also important to note that using the ‘mount –o discard’ option may affect overall performance – again refer to your Linux Vendor for their recommendations.

Automatic reclamation

Since SANsymphony-V 9.0 PSP4 the DataCore Server will, in the background, continuously scan each Physical Disk in a Pool for any SAUs that contain all-zero data which are then reclaimed back into the Disk Pool.

The Automatic Reclamation process runs at a low priority, so as to not potentially interfere with overall Disk Pool performance, and so will generally take longer to scan a Physical Disk in a Pool compared to a Manual Reclamation request (see below). However no user intervention is required.

Manual reclamation

For Linux Hosts that either do not support the fstrim command, the ‘mount –o discard’ option, or are using Virtual Disks served from DataCore Servers running SANsymphony-V 9.0 PSP3

(25)

Appendix C – Reclaiming storage

update2 or earlier (including all versions of SANsymphony-V 8.x); a suggestion would be to create a sparse file of an appropriate size (If there is enough free space available in the file system) and then zero-fill it using the dd command.

This example creates an ‘empty’ file (called ‘my_file’) of 2GB in size:

fallocate --length 2147483648 my_file

Then write zeroes within the file using dd:

dd if=/dev/zero of=my_file bs=1024 count=2097152

Then either wait for an Automatic Reclamation to take place or run a Manual Reclamation. See the Performing Reclamationsection from the SANsymphony Help:

http://www.datacore.com/SSV-Webhelp/Reclaiming_Virtual_Disk_Space.htm Note that it is also possible to script manual reclamation using the Start-DcsVirtualDiskReclamation PowerShell Command.

How much storage will be reclaimed?

It is impossible to predict exactly how many Storage Allocation Units (SAUs) will be reclaimed. For reclamation of an SAU to take place, it must contain only ‘all-zero’ block data over the

entire SAU else it will remain allocated and this is entirely dependent on how and where the Host has written its data on the DataCore LUN.

For example, if the Host has written the data in such a way that every allocated SAU contains a small amount of non-zero block data, even if the total amount of data is significantly less than the total amount of SAU allocation, then no SAUs can be reclaimed.

It may be possible, in some cases, to use the Host Operating System’s own defragmentation

tools to force the non-zero block data to be moved to a more contiguous pattern, so leaving the ‘end’ of the DataCore LUN full of SAUs that no longer have non-zero data on them that can then be reclaimed. However care should be taken that the act of defragmenting the data does not itself cause more SAU allocation as the block data is moved around during the re-organization. DataCore Software can offer no guarantees.

(26)

Previous changes

2016

April Added

Linux compatibility list

Red Hat Enterprise Linux Versions 7.1 – 7.2

SUSE Linux Enterprise Server 12 (no Service Pack) and 12 Service Pack 1 January

Updated

Red Hat Enterprise Linux Version 5.5 – 5.10

Previously this version was listed as 'not qualified' with ALUA enabled Hosts and SANsymphony-V 9.x. This has now been changed to 'Not Supported with Mirrored or Dual Virtual Disks'.

Red Hat Enterprise Linux Version 6.6

Previously this version was listed as 'Not Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x. This version is now 'Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x.

2015

November Added

Linux Applications – SAP HANA

Including an example of a 3-node SAP HANA configuration.

Updated

SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now ‘End of Life’. Please see: End of Life Notifications http://datacore.custhelp.com/app/answers/detail/a_id/1329

September Added

Linux Applications – SAP HANA

A new section with settings specific to SAP HANA.

Updated

SUSE Linux Enterprise Server 11.0 Service Pack 3 using ALUA are certified with SAP HANA versions SPS09 and SPS10.

Multipath Configuration Settings

The multipath.conf settings for all qualified Linux distributions are now identical so there is only one section for all Linux distributions listed here.

The setting dev_loss_tmo infinity requires the additional setting fast_io_fail_tmo to be present in the

multipath.conf file – this was omitted previously from the SUSE Linux Enterprise Server requirements; and that the fast_io_fail_tmo value is set to 5 – this was previously set to 30 for Red Hat Enterprise Linux requirements.

(27)

Previous changes

June Added

Ubuntu 14.04 LTS has now been qualified with SANsymphony-V 10.x.

Known Issues

Ubuntu requires manual intervention to redetect failed paths to a DataCore Server.

April Added Known Issues

All RHEL or SUSE Linux specific ‘known issues’ with SANsymphony-V will now be documented here.

March Updated

SUSE Linux Enterprise Server 11.0 with Service Pack 3 is now qualified with SANsymphony-V 10.x using ALUA-only

settings.

February Updated

Red Hat Enterprise Linux 7.0 is now qualified with SANsymphony-V 10.x using ALUA-only settings.

Multipath Configuration Settings – Red Hat Enterprise Linux

Added new entry in the multipath.conf file

no_path_retry fail

This is required for any Oracle RAC/GFS configured Hosts

Hosts using ALUA with the ‘Preferred Server’ setting configured for ALL.

2014

November Updated

Updated the table for all Red Hat Enterprise Linux and SUSE Linux Enterprise Server Versions

Single Virtual Disks are now always considered supported.

July Updated

Updated the table for all Red Hat Enterprise Linux Versions 5.0 - 5.10. It was previously listing versions 5.0 – 5.2 only.

Host Settings

Disk Timeouts – Added an example for use of the cat command to determine the current SCSI Disk Timeout and a note that some versions of Linux may revert to the default settings after a reboot and how to resolve this.

(28)

Previous changes

June Updated

Updated the table for SANsymphony-V 10.x

May Added

Red Hat Enterprise Linux – 6.5 with ALUA is now qualified for use with SANsymphony-V 9.x

Appendix A - Reclaiming Storage from Linux Hosts

Added information for ATA Trim commands and Automatic Reclamation from Disk Pools.

Updated

Multipath Configuration Settings Red Hat Enterprise Linux only

Added a new Multipath.conf entry requirement - fast_io_fail_tmo - which is required to support the dev_loss_tmo

value of ‘infinity’ else the value may default back to 10 minutes instead. NB: Check with your Vendor to make sure your specific Linux Kernel version supports these options.

Red Hat Enterprise Linux and SUSE Enterprise Linux

DataCore recommends that the Multipath.conf entry path_grouping_policy be ‘group_by_prio’ instead of ‘failover’. NB: The ‘failover’ option is considered unqualified for RHEL version 6.5.

Added an optional Multipath.conf entry user_friendly_names which will create simpler, easier to read, disk device names for users to work with. NB: Check with your Vendor to make sure your specific Linux Kernel version supports this option.

April

This document combines all of DataCore’s Linux-related information from older Technical Bulletins into a single document including:

Technical Bulletin 2a: "‘Other’ Linux Hosts"

Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts"

Technical Bulletin 8: "Formatting Host’s File Systems on Virtual Disks created from Disk Pools" Technical Bulletin 11: "Disk Timeout Settings on Hosts"

Technical Bulletin 16: "Reclaiming Space in Disk Pools" Added

Added new tables to show which versions are explicitly qualified, unqualified and not supported with either SANsymphony-V 8.x or 9.x, and if the configuration is with or without ALUA enabled Hosts. Note that the minimum requirement for SANsymphony-V 8.x is now 8.1 PSP1 Update 4 to guarantee expected behavior for qualified versions of Linux.

Appendix A

This section gives more detail on the Preferred Server and Preferred Path settings with regard to how it may affect a Host.

(29)

Previous changes

This section incorporates information regarding "Reclaiming Space in Disk Pools" (from Technical Bulletin 16) that is specific to Linux Hosts.

Updated

Host Settings - SUSE Linux Enterprise Server with/without ALUA enabled

dev_loss_tmo - For SLES 11 SP2 or greater, please make sure that multipath-tools-0.4.9-0.70.72.1 or greater is installed for the ‘infinity’ setting to work properly.

Improved explanations to most of the required Host Settings and DataCore Server Settings generally.

2013 and earlier

Technical Bulletin 2a: "‘Other’ Linux Hosts" July 2013

Removed all references to SANmelody as this is now ‘End of Life’ as of December 31 2012. Completely updated what is considered ‘qualified’, ‘not qualified’ and ‘not supported’ in this document.

January 2013

Updated the section for the multipath.conf sections specifically for the polling_interval line.

July 2012

Added SANsymphony-V 9.x. Updated the ‘Notes’ section for the multipath.conf sections specifically for the getuid_callout line.

May 2012

Updated DataCore Server and Host minimum requirements. Removed all references to ‘End of Life’ SANsymphony and SANmelody versions that are no longer supported as of December 31 2012. Updated the configuration settings and entries for /etc/multipath.conf (with additional values and explanations) for all DataCore products. Users should re-check their existing configurations and make any appropriate changes. Removed all (out of date) step-by-step instructions on how to manage and configure/format disk devices on the Host.

March 2011

Added SANsymphony-V

December 2009

Initial Publication

Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" July 2013

Removed all references to RHEL version 5.3 as this were never tested with ALUA-enabled hosts and so may cause confusion. Please use Technical Bulletin 2a for this earlier version. This Bulletin is now only for RHEL versions 6.x. Updated Host Requirements. Added additional information in the ‘Notes’ section of the multipath.conf section.

June 2013

Improved blacklist_exceptions example for multipath.conf.

April 2013

Removed all references to SANmelody as this is now ‘End of Life’ as of December 31 2012. Updated settings for multipath.conf including additional settings and explanations.

(30)

Previous changes

January 2013

Updated the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead.

Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts" July 2013

This Bulletin is now only for SLES versions 11.x. Updated Host Requirements. Completely updated what is considered ‘qualified’, ‘not qualified’ and ‘not supported’ in this document. Added additional information in the ‘Notes’ section of the multipath.conf section.

June 2013

Improved blacklist_exceptions example for multipath.conf.

April 2013

Removed all references to SANmelody as this is now ‘End of Life’ as of December 31 2012. Updated settings for multipath.conf including additional settings and explanations.

February 2013

Added notes for SLES 11 SP2; including an additional multipath.conf setting. Updated that SLES 11 SP1 is no longer supported as this was found not work correctly in all failure conditions. Users on SLES 11 SP1 should update to SP2.

January 2013

Updated the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead.

(31)

COPYRIGHT

DataCore, the DataCore logo and SANsymphony are trademarks of DataCore Software Corporation. Other DataCore product or service names or logos referenced herein are trademarks of DataCore Software Corporation. All other products, services and company names mentioned herein may be trademarks of their respective owners.

ALTHOUGH THE MATERIAL PRESENTED IN THIS DOCUMENT IS BELIEVED TO BE ACCURATE, IT IS PROVIDED “AS IS” AND USERS MUST TAKE ALL RESPONSIBILITY FOR THE USE OR APPLICATION OF THE PRODUCTS DESCRIBED AND THE INFORMATION CONTAINED IN THIS DOCUMENT. NEITHER DATACORE NOR ITS SUPPLIERS MAKE ANY EXPRESS OR IMPLIED REPRESENTATION, WARRANTY OR ENDORSEMENT REGARDING, AND SHALL HAVE NO LIABILITY FOR, THE USE OR APPLICATION OF ANY DATACORE OR THIRD PARTY PRODUCTS OR THE OTHER INFORMATION REFERRED TO IN THIS DOCUMENT. ALL SUCH WARRANTIES (INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, FITNESS FOR A PARTICULAR PURPOSE AND AGAINST HIDDEN DEFECTS) AND LIABILITY ARE HEREBY DISCLAIMED TO THE FULLEST EXTENT PERMITTED BY LAW.

The Host Server. Linux Configuration Guide. July The Data Infrastructure Software Company

The Host Server

Linux Configuration Guide

Table of contents

Overview

Recent changes made to this document

Previous changes made to this document

Linux compatibility lists

RedHat Enterprise Linux

Compatibility notes

Fibre Channel and iSCSI Connections

SCSI UNMAP

Linux compatibility lists

SUSE Linux Enterprise Server

Compatibility notes

Fibre Channel and iSCSI Connections

SCSI UNMAP

Linux compatibility lists

Ubuntu

Compatibility notes

Fibre Channel and iSCSI connections

SCSI UNMAP

Linux compatibility lists

Compatibility notes

Other Linux distributions

Self-qualifying 'other' Linux distributions

The DataCore Server's settings

Operating system type

Port roles

Multipathing support

The DataCore Server's settings

Asymmetrical Logical Unit Access (ALUA) support

Serving Virtual Disks to the Hosts for the first time

Virtual Disks LUNs and serving to more than one Host or Port

The Linux Host's settings

Operating system settings

The Linux Host's settings

Multipath configuration settings

The Linux Host's settings

The Linux Host's settings

Device section notes

The Linux Host's settings

Linux applications

SAP HANA

Sizing guidelines

Linux Applications – SAP HANA

Linux Applications – SAP HANA

Known issues

Known issues

Appendix A

Preferred Server & Preferred Path settings

Without ALUA enabled

With ALUA enabled

Appendix A - Preferred Server & Preferred Path settings

Appendix B

Configuring Disk Pools

Appendix C

Reclaiming storage

Using SCSI UNMAP commands

Automatic reclamation

Manual reclamation

Appendix C – Reclaiming storage

How much storage will be reclaimed?

Previous changes

2016

2015

Previous changes

2014

Previous changes

Previous changes

2013 and earlier

Previous changes