Performance Analysis and Testing of Storage Area Network

Performance Analysis and Testing

of Storage Area Network

Yao-Long Zhu, Shu-Yu Zhu and Hui Xiong

Data Storage Institute, Singapore

Email:

[email protected]

Motivations

Router Router WAN FCIP FC/DWDM FCIP FC/DWDM Block level IP protocol File level FC protocol iSCSI iSCSI LAN Clients Server NAS FC-SAN Storage system IP storage system Clients Storage system LAN Server NAS FC-SAN IP storage system

•What should we do to optimize the storage system directly connected to storage network instead of directly connected to server?

•How to compare IP storage, Fibre Channel, and InfinBand?

•Do we need new algorithms to replace the RAID technology which introduced in 1980s?

•How to evaluate and analyze the performance of the Storage Area Network easily and quickly?

Key Points

Build up a Queuing Network Model for

total SAN

performance analysis

from perspective of networking

Ÿ

Analyze the effects of the I/O workload on the SAN performance

Ÿ

Analyze the effects of disk cache and Fork/Join model

Ÿ

Analyze the FC-AL’s scheduling algorithms

Comparison of the theoretical and experimental

results

Modeling of SAN

Server

Server

Disk

Disk

FC-AL

FC-switch

RAID

controller

Client

Client

IP switch

Disk

Hosts FC-SW Network

λ

_host1

λ

_fc-sw

λ

host2 Disk Array Controller and Cache FC-AL Network

λ

da

λ

disk

λ

_fc-al HDA center Disk Controller & Cache

λ

hda Disk Unit Disk Array

Queuing Network Model

Server Server Disk Disk Disk FC-AL FC-switch RAID controller Client Client IP switch IO Queue

•Network’s view points

•I/O response time

•Throughput (MB/s)

0 20 40 60 80 100 120 140 160 180 0 10 20 30 40 50 60 70 80 90 SAN performance (MB/s) Response time (ms) HDA Diskcache FC-AL dacc FCF Host 0 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100

Request rate (IOPs)

Utilizations Host FCF dacc FC-AL Diskcache HDA

SAN Performance for Big Sequential I/O

System response time varies with system throughput for sequential 1MB I/O size.

Utilizations of service nodes vary with system request rate for sequential 1Mb I/O size.

I/O workload: read System configuration:

1 server+ 1 daccDACC + 5 FC-AL disks HDA: Head-Disk Assembly (mechanical part) dacc: Disk array controller and cache

FCF: FC fabric Switch

0 20 40 60 80 100 120 140 0 10 20 30 40 50 60 SAN performance (MB/s) Response time (ms) HDA Diskcache FC-AL dacc FCF Host

SAN Performance for Big Random I/O

0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 70 request rate(IOPs) Utilizations Host FCF dacc FC-AL Diskcache HDA

HDA response time is the largest portion (40%~60%) in the whole response time distribution.

performance of

5 hard disks : 66MB/s

15 Hard disks : 79MB/s 25 Hard disks : 82MB/s

System response time varies with system throughput for random 1MB I/O size.

Utilizations of service nodes vary with system request rate for random 1MB I/O size

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0 2 4 6 8 10 12 14 16 18 System performance (MB/s) response time (ms) HDA Diskcache FC-AL dacc FCF Host 0.0 0.2 0.4 0.6 0.8 1.0 0 1000 2000 3000 4000 5000

Request rate (IOPs)

Utilizations Host FCF dacc FC-AL Diskcache HDA

System response time varies with the throughput for sequential 4KB I/O (single user)

SAN Performance for Small Sequential I/O

0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 6 12 18 24 30 36 42 48 SAN performance (MB/s) Response time (ms) HDA Diskcache FC-AL dacc FCF Host 0.0 0.2 0.4 0.6 0.8 1.0 0 2000 4000 6000 8000 10000 12000 14000 Request rate (IOPs)

Utilizations Host FCF dacc FC-AL Diskcache HDA

Utilizations vary with system request rate for sequential 4KB I/O (single user). (4500 IOPs)

System response time varies with the throughput for sequential 4KB I/O (2 users).

Utilizations varies with the system request rate for sequential 4KB I/O (2 users). (5600 IOPs)

Disk Array Controller = Bottleneck !

Host Server = Bottleneck !

0 10 20 30 40 50 60 70 0.0 0.4 0.8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 4.0 4.4 4.8 SAN performance (MB/s) Response time (ms) HDA Diskcache FC-AL dacc FCF Host

SAN Performance for Small Random I/O

0 10 20 30 40 50 60 70 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 performance(MB/s) response time (ms) HDA Diskcache FC-AL dacc FCF Host 0 0.2 0.4 0.6 0.8 1 1.2 0 100 200 300 400 500 Request rate(IOPs) Utilizations Host FCF dacc FC-AL Diskcache HDA

System response time varies with the throughput for

random 4KB I/O (5 HDDs). Utilizations vary with the system request rate for _{random 4KB I/O (5 HDDs).}

System response time varies with the throughput

Performance of

5 hard disks : 420 IOPs 15 hard disks : 1260 IOPs 25 hard disks : 2100 IOPs

Empirical Model Comparison

0

20

40

60

80

100

1

2

4

8

16

32

64

128 256 512 1024

I/O size (KB)

Throughput (MB/s)

theoretical

experimental

The tests are based on the multiple Pentium 733MHz hosts with 64bits and 66MHz PCI bus, HBA with Qlogic QLA 2200A, 1G FC switch with Brocade Silkwarm 2400, and a self-developed virtual FC Disk. IOMeter is used as the benchmark tool. Both of the testing and theoretic results show that the FC network is the system bottleneck for big I/O size (>32KB), and the storage system controller overhead is the system limitation for small I/O size (<16KB).

SAN performance varies with I/O size for sequential I/O workload

Lesson #1:

Disk Cache / Controller Performance

Response time varies with I/O size when the request rate is 200 IOPs for hard disk.

0

2

4

6

8

1

2

4

8

16

32

64

128

I/O size (KB) HDA DCC 0 5 10 15 20 25 30 35 40 45 1 2 4 8 16 32 64 128 256 512 1024 IO size (KB) Throughput( MB/s)

Disk HDA Cache

Bandwidths of the disk unit, DCC and HDA vary with IO size for single hard disk.

λ

_disk

HDA center

Controller and Cache center

λ

_hda

Hard Disk model

Single Disk – Cache MAY be bottleneck Disk Array – Cache depends largely on Disk Stripe Size

Small I/O : DCC overhead larger than Physical Drive overhead Large I/O : Physical Drive overhead dominates

Lesson #2:

Disk Array and Fork/Join Model

1 1.1 1.2 1.3 1.4 1.5 0 10_{Disk number}20 30 Ratio a c c e s s t i m e waiting time

Fork/Join model

disk disk disk disk disk k k TS E TS TR λ µ ρ µ δ λ − + = (0) * * * [ ] ) ( 2 2 Request j Request j for disk 1 Request j out Request j for disk k Disk Array Controller and Cache FC-AL Network λda λdisk λdisk λdisk λ_fc-al λ_disk Disks

Fork/Join access time and queue waiting time ratio vary with disk numbers.

Response time ratio varies with disk numbers and system utilization.

Lesson #3:

FC-AL Algorithms

0

1

2

3

4

0

0.2

0.4

0.6

0.8

1

FC-AL utilization

Data waiting time (ms)

FAA R C F FLF C F

0

1

2

3

4

0

0.2

0.4

0.6

0.8

1

FC-AL utilization

Cmd waiting time (ms) FAA RCF FLF CF

Data average waiting time varies with FC-AL utilization.

Command average waiting time varies with FC-AL utilization

FAA: Fairness Access Algorithm RCF: Read Command First

FLF: FL-port First CF: Commands First

Lastly….

The queuing network model has been shown to be a useful

tool for analyzing the overall SAN system performance

The model was also used to analyze our advanced SAN

storage technology, DA

2

We missed the proceedings publication. However, this

paper will be made available on the conference website!

More Questions? Visit our Poster Session