UNH-IOL iscsi and DATA CENTER BRIDGING CONSORTIUM

(1)

UNH-IOL

iSCSI and

DATA CENTER BRIDGING

CONSORTIUM

iSCSI over DCB Interoperability Test Suite

Version 1.2

Technical Document

Last Updated: July 25, 2012

Data Center Bridging and iSCSI Consortium 121 Technology Drive, Suite 2

InterOperability Laboratory Durham, NH 03824

University of New Hampshire Phone: (603) 862-0701

Fax: (603) 862-4181

(2)

MODIFICATION RECORD

April 26, 2011 (Version 1.0) Mikkel Hagen

August 3, 2011 (Version 1.1) Fixed the large I/O size July 25, 2012 (Version 1.2) Fixed the ETS test procedure

(4)

ACKNOWLEDGMENTS

The University of New Hampshire would like to acknowledge the efforts of the following individuals in the development of this test suite.

University of New Hampshire Mikkel Hagen University of New Hampshire Robert Noseworthy University of New Hampshire Peter Scruton University of New Hampshire Ryan Goulding

(5)

INTRODUCTION

Overview: The University of New Hampshire’s InterOperability Laboratory (IOL) is an institution designed to improve the interoperability of standards based products by providing an environment where a product can be tested against other implementations of a standard. This suite of tests has been developed to help vendors evaluate the functioning of their iSCSI over DCB based products. Rather, they provide one method to isolate problems within a iSCSI over DCB device. Successful completion of all tests contained in this suite does not guarantee that the tested device will operate with all other iSCSI over DCB devices. However, combined with satisfactory operation in the IOL’s semi-production environment, these tests provide a reasonable level of confidence that the Device Under Test (DUT) will function well in most multi-vendor iSCSI over DCB environments.

Test Number and Title: The test number is given based on the order of the test within the test group. Groups are arranged according to similar test setups or similar observable results. The title is a basic description of the test.

Purpose: The purpose is a short statement describing what the test attempts to achieve. The test is written at the functional level.

Resource Requirements: The requirements section specifies the test hardware and/or software needed to perform the test. This is generally expressed in terms of minimum requirements, however in some cases specific equipment manufacturer/model information may be provided.

Last Modification: This specifies the date of the last modification to this test.

Discussion: The discussion covers the assumptions made in the design or implementation of the test, as well as known limitations. Other items specific to the test are covered here as well.

Test Setup: The setup section describes the initial configuration of the test environment. Small changes in the configuration should not be included here, and are generally covered in the test procedure section (next).

Procedure: The procedure section of the test description contains the systematic instructions for carrying out the test. It provides a cookbook approach to testing, and may be interspersed with observable results.

Observable Results: This section lists the specific observables that can be examined by the tester in order to verify that the DUT is operating properly. When multiple values for an observable are possible, this section provides a short discussion on how to interpret them. The determination of a pass or fail outcome for a particular test is generally based on the successful (or unsuccessful) detection of a specific observable.

Possible Problems: This section contains a description of known issues with the test procedure, which may affect test results in certain situations. It may also refer the reader to test suite appendices and/or other external sources that may provide more detail regarding these issues.

(6)

TEST SETUPS

Screening Test Setup: The Target and the Initiator are connected through the Switch.

Test Setup: The Targets and the Initiators are connected through the line of Switches.

CNA

Server

iSCSI over

DCB Target

iSCSI over

DCB Target

T(1)

iSCSI over

DCB Target

T(2)

iSCSI over

DCB Target

T(3)

CNA

I(1)

Server

CNA

I(2)

Server

CNA

I(3)

Server

(7)

Group 1: Screening

Overview: These tests are designed to verify basic interoperability of different Targets with the Initiators. The following tests examine the behavior of an iSCSI over DCB Initiator device that is operating on a fabric with a single Target.

The interaction between an iSCSI over DCB Initiator and a Target is complex. This interoperability procedure is designed to verify that each iSCSI over DCB Initiator works with at least a single iSCSI over DCB Target. The procedure requires the ability to observe the behavior of the devices, and to understand the proper behavior of the traffic between the devices.

The primary observable used to determine the “pass/fail” result of test cases is the successful operation of the iSCSI-based traffic. During each step, when issues are seen a trace is made and then examined. If the trace shows improper or unexpected behavior the tester may label the test as failing.

iSCSI over DCB Targets are identified by the following notation: T(n). Each iSCSI over DCB Target is tested in a pair configuration with an Initiator noted as I(n). The Target T(n) and Initiator I(n) are connected through the Switch. The monitor is configured so that the input to the Initiator and the output from the Initiator can be observed. This screening process is then performed for each iSCSI over DCB Initiator/Target.

This testing document describes an interoperability procedure. Interoperability procedures require a high degree of analysis by the tester to ensure that the behavior is correct. For this reason the observable results section should be used as a guide. The guide may need to be modified to take into account specifics of the equipment being used in the testing.

(8)

Test 1.1: iSCSI Login

Purpose: To verify proper iSCSI login between each Initiator and Target.

Resource Requirements:

• Monitor to capture traffic.

• Applications or system software that allows the administrator to configure Target resources.

Modification Record: April 26, 2011

Discussion: The screening process is designed to ensure that each device is within a limit of capability before entering into the main test process. Each device is validated with one set of devices to verify that later tests have a reasonable chance of success. Passing devices in the screening process indicates which devices will be used in the later main procedure of testing. DCB parameters are expected to be configured on the switch and accepted by the end nodes via DCBX negotiation. Configured parameters:

• iSCSI VLAN ID – 1002

• Lossless iSCSI Priority – 4

• non-iSCSI VLAN ID – 1

• Lossless non-iSCSI Class Priority – 5

• Lossy non-iSCSI Class Priority – 2

Test Setup: Screening Setup. The Initiator and Target are connected to the Switch. All devices are powered on and configured as above.

Procedure:

1. Instruct the initiator to discover the target. 2. Instruct the initiator to login to the target.

Observable Results:

• Verify that proper DCBX negotiation occurs between each device including: PFC TLV and iSCSI App

TLV.

• Verify that all devices can properly communicate over each of the three VLAN IDs configured. • Verify that the Target and all of its drives appear in the management of the Initiator.

(9)

Test 1.2: iSCSI Read

Purpose: To verify proper iSCSI read operations between each Initiator and Target.

Modification Record: August 3, 2011

Test Setup: Screening Setup. The Initiator and Target are connected to the Switch. All devices are powered on and configured as above. The initiator has discovered and logged into the target and all of its drives.

Procedure:

1. Instruct the initiator to perform 100% sequential read operations using 512B SCSI requests for 3 minutes. 2. Instruct the initiator to perform 100% sequential read operations using 512KB SCSI requests for 3 minutes. 3. Instruct the initiator to perform 100% sequential read operations using 1MB SCSI requests for 3 minutes.

• Verify that all read operations complete successfully between the initiator and the target.

(10)

Test 1.3: iSCSI Write

Purpose: To verify proper iSCSI write operations between each Initiator and Target.

Procedure:

1. Instruct the initiator to perform 100% sequential write operations using 512B SCSI requests for 3 minutes. 2. Instruct the initiator to perform 100% sequential write operations using 512KB SCSI requests for 3

minutes.

3. Instruct the initiator to perform 100% sequential write operations using 1MB SCSI requests for 3 minutes.

• Verify that all write operations complete successfully between the initiator and the target.

(11)

Test 1.4: iSCSI Read/Write

Purpose: To verify proper iSCSI read and write operations between each Initiator and Target.

Procedure:

1. Instruct the initiator to perform 50% sequential read operations using 512B SCSI requests and 50% sequential write operations for 3 minutes.

2. Instruct the initiator to perform 50% sequential read operations using 512KB SCSI requests and 50% sequential write operations for 3 minutes.

3. Instruct the initiator to perform 50% sequential read operations using 1MB SCSI requests and 50% sequential write operations for 3 minutes.

• Verify that the mixed traffic operations complete successfully between the initiator and the target.

(12)

Group 2: Fabric Build

Overview: The devices which were deemed passing during Group 1 screening testing are added to the fabric with a single iSCSI over DCB initiator and target pair one at a time. After the addition of each new pair of devices the stability of the fabric is tested.

(13)

Test 2.1: Fabric Build

Purpose: To verify proper operations of the Targets and the Initiators during fabric build.

• Applications or system software that allows the administrator to:

o Configure Target resources.

o Perform read/write operations to Target resources.

Discussion: In the fabric build phase devices that have operated properly in the screening phase are added to a fabric one at a time. After the addition of each pair of devices the fabric is examined to see if it is still operating properly. iSCSI traffic is checked or started to verify proper operation of the fabric. During the fabric build phase of testing it may be necessary to eliminate devices. A device is removed from the build if the device does not perform properly or exhibits some other behavior that causes other devices to malfunction. In the case where two incorrect behaviors are observed (i.e. a device creates an error condition on the fabric and another device does not respond properly to the error condition) the device which generated the error condition will be removed.

Test Setup: Test Setup. The three switches are connected to each other and configured.

Procedure:

For each Initiator/Target pair to be added one at a time to the fabric repeat the following procedure:

1. Ensure the fabric is in a stable state with iSCSI traffic between each Initiator and all Targets currently on the fabric.

2. Connect and then power on the next Target and wait until the Target is operational. 3. Connect and then power on the next Initiator and wait until the Target is operational.

4. Start a mix of iSCSI read/write activity from the new Initiator to all existing Targets if the fabric appears to be operational.

• iSCSI traffic on previously added devices continues to operate properly.

• Added Initiator/Target pair establishes connections with all other devices on the fabric. • Added Initiator/Target pair's traffic is properly operational.

(14)

Group 3: Full Fabric Tests

Overview: The functioning fabric consisting of each iSCSI Initiator and each iSCSI Target connected through a line of DCB switches is subjected to various I/O loads and disruptions including cable disconnects.

(15)

Test 3.1: Read Operations

Purpose: To verify proper operations of the Initiators and the Targets during SCSI Read operations. Resource Requirements:

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read operations to all targets.

Test Setup: Test Setup. The fabric is configured and operational.

Procedure:

1. Instruct all Initiators to discover each target and all drives in each target.

2. Instruct each Initiator to perform 100% Read operations of 512B for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

3. Instruct each Initiator to perform 100% Read operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

4. Instruct each Initiator to perform 100% Read operations of 1MB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

• Verify that Targets and all drives show up in the management of each host initiator system.

• Verify that the 5 minutes of Read operations completes successfully between the Initiators and the Targets.

(16)

Test 3.2: Write Operations

Purpose: To verify proper operations of the Initiators and the Targets during SCSI Write operations.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Write operations to all targets.

Procedure:

2. Instruct each Initiator to perform 100% Write operations of 512B for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

3. Instruct each Initiator to perform 100% Write operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

4. Instruct each Initiator to perform 100% Write operations of 1MB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

• Verify that all Targets and all drives show up in the management of each host initiator system.

• Verify that the 5 minutes of Write operations completes successfully between the Initiators and the Targets.

(17)

Test 3.3: Read and Write Operations

Purpose: To verify proper operations of the Initiators and the Targets during SCSI mixed Read and Write operations.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read and Write operations to all targets.

Procedure:

2. Instruct each Initiator to perform 50% Read and 50% Write operations of 512B for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

3. Instruct each Initiator to perform 50% Read and 50% Write operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

4. Instruct each Initiator to perform 50% Read and 50% Write operations of 1MB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

• Verify that the 5 minutes of Read and Write operations completes successfully between the Initiators and the Targets.

(18)

Test 3.4: Initiator Disconnect

Purpose: To verify proper operations of the Initiators and the Targets during disruptions such as cable pulls.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read and Write operations to all targets while cable pulls on the initiator side are performed.

Procedure:

3. While the Read and Write operations are ongoing, disconnect and reconnect initiator I(1) from the fabric. After the cable is reconnected, instruct initiator I(1) to rediscover each target and all drives.

4. Instruct initiator I(1) to begin 50% Read and 50% Write operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

5. Repeat Steps 2 – 4 for initiators I(2) and I(3).

• Verify that the 5 minutes of Read and Write operations in Step 4 completes successfully between the Initiators and the Targets.

• Verify that traffic is not disrupted when an initiator is removed from the fabric and then reconnected.

(19)

Test 3.5: Target Disconnect

Purpose: To verify proper operations of the Initiators and the Targets during disruptions such as cable pulls.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read and Write operations to all targets while cable pulls on the target side are performed.

Procedure:

3. While the Read and Write operations are ongoing, disconnect and reconnect target T(1) from the fabric. After the cable is reconnected, instruct each initiator to rediscover target T(1) and all drives.

4. Instruct each initiator to begin 50% Read and 50% Write operations of 512KB for 5 minutes to all target T(1) drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

5. Repeat Steps 2 – 4 for targets T(2) and T(3).

• Verify that the 5 minutes of Read and Write operations in Step 4 completes successfully between the

Initiators and the Targets.

• Verify that traffic is not disrupted when a target is removed from the fabric and then reconnected.

(20)

Test 3.6: Converge Two Streams

Purpose: To verify proper operations of the Initiators and the Targets while iSCSI traffic is converged with non-iSCSI traffic.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read and Write operations to all targets while non-iSCSI traffic begins to be transmitted on the fabric.

Test Setup: Test Setup. The fabric is configured and operational. Each switch's ETS settings are configured to provide 50% bandwidth to iSCSI traffic class and 50% traffic to non-iSCSI traffic. A traffic generator is connected to the switch in the middle of the fabric.

Procedure:

3. During the Read and Write operations begin to source non-iSCSI, broadcast, maximum sized frame traffic from the traffic generator at line rate.

4. Repeat steps 2-3 instructing each Initiator to perform 100% Write operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

• Verify that the 5 minutes of Read and Write operations completes successfully between the Initiators and

the Targets.

• Verify the bandwidth of the iSCSI traffic stream and the non-iSCSI traffic stream does not fall below 40%

or go above 60% at any time.

Possible problems: Depending on observed iSCSI traffic throughput, ETS settings may need to be adjusted in order to observe proper enforcement of bandwidth allocation.

(21)

Test 3.7: Reconfigure ETS

Purpose: To verify proper operations of the Initiators and the Targets while iSCSI traffic is converged with non-iSCSI traffic and ETS bandwidth settings are reconfigured.

Discussion: During this phase of testing, the fabric is stable and all Initiators begin SCSI Read and Write operations to all targets while non-iSCSI traffic begins to be transmitted on the fabric. Once the fabric is stable, the bandwidth percentages are changed.

Test Setup: Test Setup. The fabric is configured and operational. Each switch's ETS settings are configured to provide 50% bandwidth to iSCSI traffic class and 50% bandwidth to non-iSCSI traffic. A traffic generator is connected to the switch in the middle of the fabric.

Procedure:

3. During the Read and Write operations begin to source non-iSCSI, broadcast, maximum sized frame traffic from the traffic generator at line rate.

4. After 5 minutes, reconfigure the ETS settings of each switch to provide 75% bandwidth to iSCSI traffic class and 25% bandwidth to non-iSCSI traffic.

5. Repeat steps 2-4 instructing each Initiator to perform 100% Write operations of 512KB for 5 minutes to all drives simultaneously. The 5 minutes of data may be any pattern: random, constant or a looped pattern.

• Verify that the 5 minutes of Read and Write operations completes successfully between the Initiators and

the Targets.

• Verify the bandwidth of the iSCSI traffic stream and the non-iSCSI traffic stream does not fall below 40%

or go above 60% while the switches are configured to 50/50.

• Verify after the bandwidth reconfiguration that iSCSI traffic is within +/-10% of 75% and the non-iSCSI

traffic is within +/-10% of 25%.

Possible problems: Depending on observed iSCSI traffic throughput, ETS settings may need to be adjusted in order to observe a change in throughput.

(22)

Test 3.8: Reconfigure Fabric

Purpose: To verify the proper operation of the fabric when it is reconfigured.

Discussion: During this phase of testing, we reconfigure the fabric so that each initiator and target talks to a different switch and repeat previous tests.

Procedure:

1. Rotate each switch right so that the left switch is now in the middle of the fabric and the right switch is now on the left side of the fabric.

2. Repeat Test 3.1 – 3.7.

3. Rotate the switches one more time so that each switch has been in each position in the fabric and repeat Test 3.1 – 3.7.

• Verify the results of Test 3.1 – 3.7 with the new configuration.

UNH-IOL iscsi and DATA CENTER BRIDGING CONSORTIUM

UNH-IOL

iSCSI and

DATA CENTER BRIDGING

CONSORTIUM

iSCSI over DCB Interoperability Test Suite

Version 1.2

Technical Document

Table of Contents

MODIFICATION RECORD

ACKNOWLEDGMENTS

INTRODUCTION

TEST SETUPS

CNA

Server

iSCSI over

DCB Target

iSCSI over

DCB Target

T(1)

iSCSI over

DCB Target

T(2)

iSCSI over

DCB Target

T(3)

CNA

I(1)

Server

CNA

I(2)

Server

CNA

I(3)

Server

Group 1: Screening

Group 2: Fabric Build

Group 3: Full Fabric Tests