» Storage Virtualization Seminar Presentation Download

(1)

Storage Virtualization Seminar

Presentation Download

SSppoonnssoorreedd BByy::

The management of storage devices is often tedious and time consuming. And as the fragile economy continues to impact everyone this year, we are all going to be required to do more with less. Storage virtualization promises to ease the headaches of increasingly complex storage systems and, ideally, will allow us to still effectively and efficiently do our jobs as IT pros with fewer resources.

This presentation highlights how virtualization has progressed from vaporware to an actual concept storage managers are using to minimize the amount of machines needed to manage, centralize data and change the economics of storage. Marc Staimer will also look at how storage virtualization makes heterogeneous storage compatible, eases data migration and enables consolidation.

(2)

Storage Virtualization

School

Presented By:

• Marc Staimer, President & CDS

• Dragon Slayer Consulting

• [email protected]

(3)

Dragon Slayer Consulting Intro

z

Marc Staimer - President & CDS

• 11+ years

Storage, SANS, Software, Networking, Servers Consults vendors (> 100)

Consults end users (> 400) Analysis at trade shows

Articles for Tech Target websites & magazines Blog

• 29+ years industry experience

Storage Virtualization School 22 Storage Virtualization School

(4)

The Socratic Test of Three

(5)

Seminar Agenda

z

Part 1

• What, where, who, when, how storage virtualization

z

Part 2

• Virtual storage in a virtual server world

z

Part 3

• Storage as a dynamic online “on-demand” resource

(6)

What I assume you know

• SAN versus File Storage

• Storage versus IP Networking

• Scalability issues

• Storage service issues

• Storage management issues

(7)

Part 1 What, Where, Who, When, Why, How

Storage virtualization

(8)

Old Man & The Toad

Storage Virtualization School 7 August 2009 Storage Virtualization School 7

(9)

Part 1 Agenda

z

What

z

Where

z

Who

z

When

z

Why

z

How

(10)

What is Storage Virtualization?

z

Abstract the storage image

• From the storage

• Different kinds of storage

SAN, NAS, Unified

• Different kinds of storage image abstraction

Virtualize, Cluster - Grid, Cloud, & Variations

(11)

• The act of abstracting, hiding, or isolating the internal

function of a storage (sub)system or service from

applications, compute servers or general network resources

for the purpose of enabling application and network

independent management of storage or data.

• The application of virtualization to storage services or

devices for the purpose of aggregating, hiding complexity

or adding new capabilities to lower level storage resources.

Storage can be virtualized simultaneously in multiple layers

of a system, for instance to create HSM like systems.

SNIA Storage Virtualization Definition

(12)

Virtualized Storage Image Abstraction

z

Abstracting the image means

• Masking storage services from applications

Provisioning

Increasing storage Additions

Filer mounting Data migration

• Between storage targets or storage tiers

Data protection

Change management

(13)

Polls: Storage Virtualization Market

z

Virtualization is neither new or strange

• Per ESG based on 2008 polls

52% have already implemented storage virtualization 48% plan to implement

(14)

IDG 2008 Virtualization Poll

(Collected Q4 2007)

Who Took the Survey (464 respondents)

IT Decision Maker IT Architect

Other

Developer

(15)

IDG: Where Are You Investing Now?

4% 9% 11% 15% 23% 25% 43% 47% 86% 96% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

No current virtualization investment IO Virtualization Application Grids File Virtualization Application Virtualization Enterprise Data Center Virtualization Storage Virtualization Desktop Virtualization Server Virtualization Currently Investing

Current Virtualization Investments

(16)

IDG: Where Investing Thru 2010

3% 0.4% 15% 18% 31% 38% 42% 53% 62% 81% 97% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Don't Know No planned virtualization investments Application Grids IO Virtualization File Virtualization Application Virtualization Enterprise Data Center Virtualization Storage Virtualization Desktop Virtualization Server Virtualization Planning to Invest

Future Virtualization Investments

(17)

Dragon Slayer Consulting Poll

(265 respondents)

8% 11% 12% 14% 22% 30% 45% 48% 56% 78% 92% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% No planned virtualization No Current virtualization Application Grids IO Virtualization File Virtualization Application Virtualization Enterprise Data Center Virtualization Storage Virtualization Desktop Virtualization Server Virtualization Have (or will) implemented virtualization

Current or Planned Virtualization

(18)

Where Storage Virtualization Occurs

z

Everywhere

• Odds are you’re storage is already virtualized to a degree

Operating systems Applications

Volume managers Hypervisors

Storage arrays – RAID NAS

Appliances Switches

Even SSDs

(19)

Volume Management

z

Server-based storage virtualization

• Abstracts block storage

(LUNs, HDD)

into virtual “volumes”

• Common to modern OS – built in

Windows Logical Disk Manager, Linux LVM/EVMS, AIX LVM,

HP-UX LVM, Solaris Solstice, Veritas Storage Foundation

z

Mostly used for flexibility

• Resize volumes

• Protect data (RAID)

• Add capacity (concatenate or expand stripe or RAID)

• Mirror, snapshot, replicate

• Migrate data

(20)

Logical Volume Managers (LVM)

Platform Volume Manager Notes

AIX Logical Volume Manager OSF LVM, no RAID 5, no copy-on-write snapshots HP-UX 9.0+ HP Logical Volume _Manager OSF LVM, no RAID 5

FreeBSD Vinum Volume Manager No copy-on-write snapshots

Linux 2.2+ Logical Volume Manager and Enterprise Volume

Management System Based on OSF LVM, no RAID 5

Solaris Solaris Volume Manager _{(was Solstice DiskSuite)} Limited allocation options, no copy-on-write snapshots AIX, HP-UX,

Linux, Solaris, Windows

Symantec Veritas Volume Manager (VxVM), Storage

Foundation Full-featured multi-platform volume manager

Windows 2000+ Logical Disk Manager Co-developed with Veritas, limited allocation options, _{copy-on-write snapshots introduced in Server 2003} Solaris, BSD,

Mac OS X

10.6+ ZFS Combined file system and volume manager

(21)

ZFS: Sun’s Super File System

z

a.k.a. “ZB file system”

•

Combined file system, LVM, disk/partition manager

•

Open source (CDDL) project managed by Sun

•

Replaces UFS (Sun), HFS+ (Apple OSX Snow Leopard Server)

•

Extensible full featured storage pools

Across systems, disks, &optimized for SSDs

•

File systems contained in “zpools” on “vdevs”

W / striping & optional RAID-Z/Z2

•

128-bit addresses mean theoretical near-infinite capacity

•

“copy-on-write” w / checksums for snapshots, clones,

authentication

(22)

ZFS Limitations

z

Adding or removing vdevs is hard/impossible

• Especially removing

z

Stacked RAID is currently not possible

z

There is no clustering

• Until Sun adds Lustre

(23)

IO Path Management Software

Virtualizing the SAN Pathing

z Virtualizes server – storage connection

•

Failover

•

Load balancing strategies

z Numerous choices

•

Veritas DMP (cross-platform, w / Storage Foundation)

•

EMC PowerPath (supports EMC, HDS, IBM, HP)

•

IBM SDD (free for IBM)

•

HDS (HDLM)

•

Microsoft MPIO (Windows, supports iSCSI & most FC)

•

VMware Failover Paths

August 2009 Storage Virtualization School 22

SD Request Request App IO Path Mgmt App SD HBA SP-A SP-B Interconnect topology

(24)

SAN Storage Virtualization

z

An abstraction layer

• Between hosts & physical storage

• That provides a single mgmt point

For multiple block-level storage devices in a SAN

And presents a set of virtual volumes for hosts to use

(25)

What does SAN Storage Virtualization

Do?

z

Aggregate storage assets into 1 image

• Manages, provisions, protects, etc.

Transforms “n” systems into slice & dice monolith Homogeneously or heterogeneously

z

Virtual LUNs

• Mapped to physical LUNs

• Can be larger than physical

Up to a Exabyte

(26)

SAN Tends to Be a Popular

Virtualization Location

z Usually requires less configuration & mgmt

•

As compared to server based

•

And it potentially works with all servers & storage

z Resides in the storage fabric

•

Appliance, storage controller, switch, & hybrid

•

Control & data path combined or split

(27)

Shared vs. Split Path

z Shared path intercepts traffic z Split path redirects traffic

Where’s my data? It’s over here! Control P ath Data P ath Where’s my data? It’s over there! Data P ath

(28)

Pros & Cons of Shared Path

Pros

z Simpler

•

Implementation

•

Operations

•

Management

•

Ease of use

Cons

z Scalability limitations

•

Units/nodes clustered

•

Performance / unit or node

•

Capacity / unit or node

z Performance hits

•

Additional latency

(29)

Pros & Cons of Split Path

Pros

z Scalability

•

Limited only by fabric

z Performance

•

Limited only by fabric

z Flexibility

Cons

z Complexity

•

Install, ops, mgmt

z SPAID

•

Limited by intelligent switch BW

•

Adds latency

Similar to shared path appliance

(30)

Split Path Notes

z

Split Path Combinations

• Switch – SPAID

a.k.a. split path architecture independent data streams Requires processing blades

• Server Software

Mgmt on appliance in fabric

Virtualization agent/driver on server/virtual server

(31)

SPAID Notes

z

Switch Centric

• Metadata & LUN map on switch

• Software & processing done on switch

• Each session is directed by switch to destination

Scalability constrained by switch processing & latency

(32)

Server Centric LVM Notes

z

LVM

• Virtualization SW, pathing SW, and/or LVM on server

• Mapping & processing performed on each server

• Each session is server controlled

• Overall management is difficult

Each server has its own management

(33)

Split Path Hybrids

z

Split Path Combinations

• Proprietary virtualization SW

A.k.a. agent and/or pathing SW

• Meta data appliance or intelligent appliance with SW

Meta data mapped from appliance

Management of software from appliance

Replication, data protection, etc. controlled from appliance

(34)

Advanced Hybrid

z

Fabric Centric Storage

•

Leverages std LVMs

Symantec Storage Foundation

OS LVMs, etc.

•

Leverages Filer (NAS heads) storage virtualizers

•

Provides proprietary virtualization SW for devices w/o software

•

Puts mgmt & advanced services on appliance

Snapshots, replication, mirroring, etc.

(35)

SAN Virtualization Products

Product Architecture Location Multi-vendor Repl. Notes

BlueArc Titan/Mercury Shared Path Controller Yes Yes Clusters up to 8 in GNS (HDS OEMs) DataCore

SANSymphony/Melody Shared Path Generic x86 appliance Yes Yes Supports FC storage, runs as virtual appliance on VMware Bycast Storage Grid Shared Path Grid of x86 appliances Yes Yes Geographically distributed cloud tech. EMC Invista Split Path x86 appliance + SPAID Yes No Primarily utilized for online data _{migration (CSCO& BRCD I-Switches)} EMC Unified NX-NS Shared Path Controller No No Up to 8 nodecluster, built-in file dedupe, _{auto tiering, thin provisioning} FalconStor NSS Shared Path Generic x86 appliance Yes Yes Optional post processing dedupe engine _{& runs as a VMware virtual appliance} HDS USP V / VM

Tagmastore Combination Shared/Split Path Controller Yes Yes Combination Enterprise controller & virtualization engine HP XP 2xxxx Combination _{Shared/Split Path Controller} Yes Yes OEM'ed Hitachi with additional HP _{software & services} IBM SVC Shared Path x86 Purpose-built _Appliance Yes Yes Supports most FC storage; large _{caches; IBM hardware} Incipient iNSP Split Path FC switch – SPAID Yes No No caching; supports Cisco FC blades LSI StoreAge SVM Split Path Combo x86 appliance + host _{SW or intelligent switch} Yes Yes No caching; split-path FC with low cost_{N_Port switch, resold by HP as SVSP} NetApp vFiler Shared Path Controller Yes Yes Active-active cluster, built-in file _{dedupe, also sold by IBM} RELDATA 9240I Shared Path X86 purpose-built storage

controller/appliance Yes Yes NAS/iSCSI that virtualizes internal SAS & external FC & SAS storage Seanodes Exanodes Shared Path VMware virtual _appliance Yes No Works w/internal & DAS storage _{converts into iSCSI SAN}

XIOtech Emprise 7000 Shared Path Controller No Yes Works only w/ISE (Intelligent Storage _Element) XIOtech ISE Age Split Path Hybrid Generic x86 appliance Yes Yes Requires XIOtech server virtualization _{agents or Symantec Storage Foundation}

(36)

SAN Virtualization Issues

z

Side effects

•

Spreading a storage pool across more RAID sets, &/or systems

Increases performance, reduces storage management It also increases probability of data loss

• Probability of 1 system going down is low

• Probability any 1 of many will fail increases rapidly (P1+ P2+ P3) – (P1* P2* P3₎

P = probability of a specific RAID group failing

(37)

Ways to Mitigate Increased Data

Failure Probabilities

1. Self-healing storage elements

2. Redundant array of intelligent RAID (RAIR)

•

RAIDing the RAID

3. More scalable individual storage systems within pool

August 2009 Storage Virtualization School 36 Each intelligent storage element

is self-healing, reducing the probability of an actual disk failure or RAID set failure to an extremely rare event

Putting the intelligent storage elements into a RAIR reduces that probability even further.

(38)

Virtual Network Attached Storage (NAS)

z NAS lends itself to virtualization

•

IP network connectivity and host processing possibilities

z Lots of file servers? Virtualize

•

Global namespace across all NAS & servers

•

Share excess capacity

•

Transparently migrate data (easier than redirecting users)

•

Reduce number of mount points

•

Tier files on large “shares” with variety of data

•

Create multiple virtual file servers

(39)

NAS Virtualization Products

Product Architecture Location Notes

AutoVirt Move/Clone/Map Split Path Windows servers Server 2003 R2, .NET. Primarily a mapping, _{data migration, & clone tool} BlueArc (also sold as

HDS HNAS) Shared Path Clustered NAS Clustered integrated NAS with global namespace EMC Rainfinity Shared Path Appliance or host SW DFS mgmt. Primarily data migration tool. Exanet ExaStore Shared Path Clustered NAS Clustered integrated NAS with global _namespace F5 Acopia Shared Path Switch - Appliance Split-path architecture, non-DFS

Microsoft DFS Split Path Host SW Windows/SMB only; Server 2008, 2003 R2+ _{enhanced management} NetApp vFiler Shared Path Active-Active _{Clustered NAS} Clustered NAS “head” with global _namespace

ONStor (LSI) GNS Shared Path Clustered NAS & DFS Combines clustered NAS with DFS into a _{single global namespace}

(40)

File Virtualization Issues

(a.k.a. Global Name Space – GNS or Global File System - GFS)

z

Appliances (x86 or switch) market success fits & starts

•

Requires a commitment to the appliance versus the NAS

•

For data protection (Snapshots, replication, etc.)

•

Another system to manage – often perceived as a point solution

•

Utilized primarily for non-disruptive data migration

z

GNS migrating into NAS systems as a feature

•

BlueArc, NetApp, OnStor (LSI), etc.

•

NAS GNS only works w/that same vendors NAS

(41)

Embedded Virtualization has

Transformed Storage Systems

z

Common in storage array controllers

• Arrays create large RAID sets

Carve out virtual LUNs for use by servers

• Controller clusters (and grids) redirect activity

Based on workload &availability

• Snapshots/mirrors & replication are common features

(42)

Newer Gen of Arrays – Usually Clustered

z

Include virtualization derived features

• Automated ILM

• Thin provisioning

• Data migration

• De-duplication

• Self configuring and/or self tuning storage

(43)

Virtual Storage Appliances (VSAs)

z

2 types

• Distributed

• Proxy

(44)

Distributed VSAs

z

Converts VM DAS into SAN

• All or some VM DAS

In a virtual storage SAN pool

• Data available even if node(s) fail

• Runs as a VM guest

Can also be a target for VMs

• On other physical machines

(45)

44 Storage Virtualization School

Proxy Virtual SANs

z

Aggregates VM DAS & SAN into storage pool

• All or some

• Runs as a VM guest

iSCSI target for internal & external VMs

• Plus other physical machines

• Low cost vs. NAS or SAN

(46)

VSA Products

Product Architecture Server Virtualization _Supported Notes

DataCore Software

SANSymphony Proxy VMware ESX & Microsoft HyperV Same as std SANSymphony that runs on an appliance ltd to 2TB FalconStor NSS VSA Proxy VMware ESX Same as IPStor that runs on an x86 _appliance HP LeftHand Networks

VSA Distributed VMware ESX Clusters up to ~100 nodes. Designed for ROBO

Seanodes Exanodes

VMware edition Distributed VMware ESX Protects data up to 16 nodes or volume failures, no replication. StorMagic SvSAN Proxy VMware ESX (Microsoft _{HyperV coming)} 2TB license is free

(47)

VSA Issues

z

Distributed VSAs

•

Requires a bit more memory & CPU cycles / server

z

Proxy VSAs

•

Requires quite a bit more memory & CPU cycles

On the target proxy Virtual SAN servers

• Should have limited add’l VM guests

z

License limits

•

Some are limited to 2TB

z

Where to use

•

Primarily small environments and/or ROBOs

z

Simplicity

•

VSAs are just about as easy as NAS

•

Utilize standard Ethernet technologies

(48)

Virtualized IO

z

Takes a very high BW pipe (10G or more)

•

Makes it appear as multiple unit & protocol types

FC SAN, TCP/IP Network, iSCSI SAN

•

Breaks it out at the switch to different networks & targets

z

Problem it solves – Fabric sprawl

z

3 types

•

Infiniband (IBA), Converged Enhanced Ethernet (CEE), MRIOV August 2009 Storage Virtualization School 47

Virtualized Pipe

TCP/IP Ethernet

FC or iSCSI SAN

(49)

Virtualized IO – 10 to 40G IBA

Standard IO

IBA Virtualized IO

(50)

Infiniband IO Virtualization Definitions

z HCA – Host Channel Adapter

•

Server adapter card

z TCA – Target Channel Adapter

•

Storage or Gateway adapter card

z Shared IO Gateway – Same as IO Virtualization

•

IBA to IP, FC, iSCSI gateway

z RDMA – Remote Direct Memory Access

•

Lowest latency memory to memory transfers

z iSCSI – IP SCSI

• SCSI mapped to TCP/IP

z HPCC – High Performance Compute Clusters

•

Large nodal clusters

z IBA Director – Large port count five 9s switch

•

288 to 864 port switches

IBA

(51)

Virtualized IO – 10G CEE

Standard IO

CEE Virtualized IO

(52)

Ethernet IO Virtualization Definitions

z FCoE – Fibre Channel over Ethernet

•

FC frames encapsulated in Ethernet packets – lightweight frame maps

z iSCSI – IP SCSI

• SCSI mapped to TCP/IP

z iWARP – RDMA on Ethernet

•

Required for HPC clusters

z CNA – Converged Net Adapters

•

Concurrent FCoE, iSCSI, iWARP, & TCP/IP on 10GbE NIC

z 10G TOE – 10G TCP offload engine

•

Provides TCP offload for 10G adapter Split stack & full stack offloads

z CEE – Converged Enhanced Ethernet

•

ANSI standard for lossless low latency Ethernet

z DCE – Data Center Ethernet

•

Cisco’s brand name for CEE

10GbE

(53)

How IBA Compares to 10GbE

InfiniBand 10GbE Notes

Max pt Bandwidth 120Gb/s 10Gb/s Faster is better

E2E latency 1 to 1.2us 10 to 50us Lower is better

Switch Latency 50 to 150ns 500ns to 10us Lower is better

RDMA Built in Voltaire only Important for _clustering

Multipath Yes Voltaire only Important for Storage

Lossless Fabric Yes Voltaire only

Power/Port 5W 15-135W Lower is better

Largest Enterprise

Switch 288 x 20Gbps &40Gbps 864

x ₂₈₈_{x 10Gbps} _{More is better}

(54)

Multi-Root IO Virtualization

z

Moving IO outside the box

(55)

MRIOV – Value Prop

Storage Virtualization School

54 August 2009 MRIOV MR OV MR OV MRIOV MRIOV MRIOV z Simpler

• Fewer storage & network pts

• Fewer storage & network switches

z Shared IO – Higher Utilization

• 75% IO cost reduction • 60% IO Power reduction • 35% Rackspace reduction z Scalable IO Bandwidth • On demand 1-40Gbps • No add’l cost

• Reduced IO adapters & cables

z Enhanced Functionality

• Shared memory

• IPC & PCIe speeds up to 40Gbps

z Reduced OpEx

• Simplified mgmt

• Server, OS, app, network, & switch transparent

• HA

• Changes w/o physical touch

z Reduced CapEx

• Smaller, denser servers

• Fewer components

• Fewer failures

(56)

How MRIOV Compares

~Significant unused IO capacity ~Inflexible, rigid server adapters ~Wasted space, cooling, power

Traditional Inefficiencies

~Flexible, IO capacity on demand ~Highest IO utilization, lowest TCO ~Standardized, open technologies

(57)

Comparison w/Other IOV Solutions

InfiniBand Solutions Today’s Solution

(No IO Virtualization) MRIOV Solution_{(PCI Express)}

Config • 2 Racks • 32 Servers • Ethernet & FC • DAS

Utilization Very Low ~15% Very High ~80%+ Low OK

Reliability Neutral High Best

Neutral

IO Perf 10Gb 80Gb 20Gb 10Gb

TCO High Low High High

Mgmt Poor Best Best OK

IO Cost High Low High High

FCoE Solutions IO power = 3000 W IO power = 700 W IO power = 2000 W IO power = 2200 W

2 Racks 1 Rack 2 Racks 2 Racks

IO Cost = $196K IO Cost = $37K IO Cost = $156K IO Cost = $180K IO components = 270 IO components = 58 IO components = 160 IO components:1 60

(58)

IO Virtualization Products

Vendor Technology Product types Notes

Aprius MRIOV Rack switch, blade switch, silicon PCIe switching w/server software

Brocade CEE CNA & CEE Top of rack switch Strong FC focus & install base, acquisition of _{Foundry provides equivalent Ethernet expertise} Cisco IBA & CEE HCA, IBA directors , switches, _{gateways, CEE top of rack switch} Ethernet leader w/strong products in FC, & IBA as _{well. Invented CEE.} Emulex CEE CNA 1 of the 2 FC HBA leaders making a strong CNA play _{(same driver interface).} Mellanox IBA & CEE HCA, TCA, IBA directors , switches, _{gateways, CNA} Dominant IBA silicon leader attempting to leverage _{position into CEE w/CNAs.} QLogic IBA & CEE HCA, TCA, IBA directors , switches, _{gateways, CNA} FC HBA leader w/strong positions in IBA silicon,HCA, TCA, directors. switches, & GWs. Invented "Shared

IO".

Virtensys MRIOV Rack switch, blade switch, silicon PCIe switching w/o server software

Voltaire IBA HCA, IBA directors , switches, _gateways IBA leader in switches, directors, gateways, _{software, & HCAs.} Xsigo IBA HCA, gateways Positions as a pure "Virtualized IO". Doesn't _{mention IBA technology.}

(59)

IO Virtualization Issues

z

IBA

•

Is primarily utilized for HPCC

•

Not a large install base in the Enterprise today

•

Few storage systems with native IBA interfac

LSI (SUN, SGI, IBM) & DDN

•

However, it is proven & it works

z

CEE

•

Technology is early & somewhat immature

•

A bit pricey (aimed at early adopters)

•

Requires new NICs & switches to be effective

z

MRIOV

•

In rack and Blade System only

•

Primarily OEM tech (e.g. must be supplied by server vendor) August 2009 Storage Virtualization School 58

(60)

Yes, Even SSDs are Virtualized

z

Virtualizes single or multi-cell flash

• Algorithms manage writes

Load balances writes across the cell or cells Makes sure SSDs have similar MTBFs as HDDs Reduces probability of flash cell write failure

(61)

SSD Tier-0 Storage Types

z

Enterprise storage

• Storage system optimized

Looks like a HDD & fits in rack Simple technology

Easy to implement – low risk

Performance constrained by storage system back end

z

Memory appliance or server adapter

• Application acceleration focused

Connects via PCIe (highest perf) (IBA, FC, 10GbE soon)

(62)

Where Storage Virtualization Occurs

z

Everywhere

• Operating systems

• Applications

• Volume managers

• Hypervisors

• Storage arrays – RAID

• NAS

• Appliances

• Switches

• Even SSDs

(63)

Audience Response

Questions?

(64)

Break sponsored

by

(65)

Part 2 Virtual storage in a virtual server world

Part 1 Part 2 Part 3 Part 1 Part 2 Part 3

(66)

Top 10 Signs You’re a Storage Admin

1. ~90%of peers & boss, don’t have a clue about what you really do 2. Being sick is defined as can't walk or you're in the hospital

3. Your relatives & family describe your job as "computer geek" 4. All real work gets started after 5pm, weekends, or holidays

5. Vacation is something you always do…next year

6. You sit in a cubicle smaller than your bedroom closet 7. Your resume is on a USB drive around your neck

8. You’re so risk averse, you wear a belt, suspenders, & coveralls

9. It's dark when you go to or leave work regardless of the time of yr. 10.You've sat at same desk for 4 yrs & worked for 3 different companies

(67)

Part 2 Agenda

z

Level setting

• For virtual servers

z

Virtual server issues

• Real world problems

z

Best practices

• To solve

(68)

Benefits of the Hypervisor Revolution

z

Increased app availability

z

Reduced server hardware w/consolidation

z

Reduced infrastructure

•

Storage network

•

IP network

•

Power

•

Cooling

•

Battery backup

z

Simplified DR

(69)

Virtualized Servers

z

Adv. features require networked storage

•

SAN or NAS

z

Virtualized server advanced functionality

•

VMware

DRS, VMotion, Storage VMotion, VDI, SRM, SW-FT, VDR, storage API,

Thin Provisioning

•

Microsoft Hyper-V

Live Migration

•

Virtual Iron

Live (Migrate, Capacity, Maint, Recovery, Convert, Snap)

•

Citrix XenServer

XenMotion, Global Resource Pooling

(70)

Source Gartner: 2008 Enterprise

Virtual Server Market Share

VMware,

87%

Citrix, 5%

Microsoft, 3% Virtual Iron, 3% SUN, 1% Oracle, 1% VMware Citrix Microsoft Virtual Iron SUN Oracle

(71)

Distributed Resource Optimization

z Distributed Resource Scheduler

• Dynamic resource pool balancing allocating on pre-defined rules

z Value

• Aligns IT resources w / bus priorities • Operationally simple

• Increases sys admin productivity • Add hardware dynamically

• Avoids over-provisioning to peak load • Automates hardware maintenance

Dynamic and intelligent allocation of hardware resources to ensure optimal alignment between business and IT

(72)

HOT! – VMotion

z Online – Increasing Data Availability

•

No scheduled downtime

•

Continuous service availability

•

Complete transaction integrity

•

Storage Network Support

iSCSI SAN, FC w / NPIV, & NAS (Specifically NFS)

(73)

Virtual Desktop Infrastructure

z Increased

•

Desktop availability, flexibility (not tied to desktop hardware), & security

z Decreased

•

Management & costs

(74)

Site Recovery Manager

z Faster more automated DR

z Integrated with storage

z Utilize lower cost DR storage & drives

Protected Primary Site Recovery / DR Site

FC Storage SATA Storage

iSCSI FCP

(75)

ESG Poll: Does Server Virtualization

Improve Storage Utilization?

Net decrease of < 20% Net decrease of 11% - 20% Net decrease of 1% - 10% No Change Net increase of 1% - 10% Net increase of 11% - 20% Net increase of > 20% 1% 4% 2% 39% 15% 21% 18%

Since being implement, what impact has server virtualization had

on your organization’s overall volume of storage capacity?

(76)

ESG Server & Storage Virtualization Poll

August 2009 Storage Virtualization School 75 7%

24%

18%

15%

36%

Has your org deployed a storage virtualization solution in conjunction with its virtual server environment?

Don't Know Yes

No, but plan to implement within next 12 mos

No, but plan to implement within next 24 mos

(77)

Why Use Virtual Storage For Virtual Servers?

z

Reasons Most Often Sited: Improved

• Mobility of virtual machines

Load balancing between physical servers

• DR & BC

• Availability

• Physical server upgradability w/o app disruptions

• Operational recovery of virtual machine images

(78)

Server Virtualization Market Trends / ESG

z Server virtualization driving storage system re-evaluation

•

66% of enterprises (>1000 employees) and 81% of small to mid sized

businesses (<1000 employees) expect to purchase a new storage system for their virtualized servers in the next 24 months.*

z IP SAN emerged as preferred server virtualization storage

•

52% of organizations deploying virtualization plan to use iSCSI

(NAS 36%, FC 27%).*

•

Clustered storage architecture advantages with server virtualization Efficient storage utilization

Optimized performance

Simpler management and flexibility More Cost effective HA/DR

* ESG, “The Impact of Server Virtualization on Storage”

(79)

What About Server Virtualization Based DR?

z DR is a prime beneficiary of server virtualization

•

Fewer remote machines idling

•

No need for identical equipment

•

Quicker recovery (RTO) through preparation &automation

z Who’s doing it?

•

26% are replicating server images, an additional 39% plan to (ESG 2008)

•

Half have never used replication before (ESG 2008)

z Based on DSC Polling

•

67% say app availability is why they implement server virtualization Justification is based on server consolidation & app up time

(80)

Server Virtualization = SAN and/or NAS

z

Server virtualization transformed the data center

•

And storage requirements

VMware #1 driver of SAN adoption today!

60% of virtual server storage is on SAN or NAS (ESG 2008) 86% have implemented some server virtualization (ESG 2008)

z

Enabled & demanded centralization

•

And sharing of storage on arrays like never before!

(81)

Types of Networked Storage

z

NAS – Network Attached Storage

•

A.K.A. File based storage

NFS – Network File System

CIFS – Computer Internet File System

• To a lesser extent AFP – Apple File Protocol

z

SAN – Storage Area Network

•

A.K.A. Block based storage

Fibre Channel (FC) iSCSI

(82)

NAS & Server Virtualization

z

NAS works very well w / Hypervisors & adv features

•

NFS – VMware, XenServer, KVM, Virtual Iron

•

CIFS – Microsoft Hyper-V

•

Common file system visible for all virtual guests

z

Incredibly simple

•

Turn it on, mount it, & you’re done

z

App performance is generally modest

•

Typically less than SAN storage

There are exceptions where it is close to equivalent

(83)

Why Hypervisor Vendors Don’t Usually

Recommend NAS

z

Performance, performance, performance

• There exception being once again BlueArc

And to a lesser extent NetApp & Exanet

Typical NAS

(84)

Virtual Servers & NAS

z

Many virtual apps are fine w / NAS performance

z

Virtual guests can boot from NFS (VMware VMDKs)

(85)

Virtual Server Issues with NAS

z

Most NAS Systems don’t

scale

well

•

Capacity, file system size, max files, & especially performance

•

Scaling typically means more systems

More systems increases complexity exponentially

• Eliminates NAS simplicity advantage

Exceptions include BlueArc, Exanet, & NetApp (GNS or GFS)

•

NAS currently does not work w/storage vMotion & SRM

(86)

SANs & Server Virtualization

z

SANs works very well w / Hypervisors & adv features

z

Mixed bag on Complexity

•

FC is very complex requiring special knowledge & skills

•

IBA is also complex & requires special knowledge & skills

•

iSCSI uses Ethernet like NAS & is almost as easy

z

App performance is very fast

•

iSCSI is fast

•

FC is a bit faster

•

IBA is fastest (albeit few choices)

z

Regardless of SAN Type

•

SANs do not overcome storage system scalability limits

(87)

Hypervisor Vendors Recommendations

z

Recommended in this order

•

iSCSI, FC, IBA

z

iSCSI Rationale

•

iSCSI is almost as fast as FC

Uses std Ethernet NICs, switches, cables & TCP/IP

•

iSCSI is almost as easy as NAS

•

iSCSI is far less expensive than FC or IBA

(88)

iSCSI SAN & Server Virtualization

z

Should have dedicated fabric

•

Not shared with other IP traffic

z

Performance mgmt

•

VLANs helps

•

QoS prioritization helps

•

1 & 10G proper utilization helps

•

1G does not require any hardware offload

•

10G may depending on performance expectations

700MB/s no offload 1GB/s with offload

(89)

FC SAN & Server Virtualization

z

FC SANs require NPIV

•

N_Port ID Virtualization

Otherwise there is an HBA / guest

Or all guests share the same WWN

z

All physical server must be in same FC Zone

•

Enables guests to still see their storage when they move

•

Critical for live migrations & business continuity

z

FCoE will have similar rules to FC

•

Difference is that it runs on 10GbE

•

Not a routed protocol, still layer 2 switching

(90)

FC SAN & Server Virtualization

z

FC SANs are manually intensive

•

Implementation, ops, change mgmt, mgmt

Software to ease burden

• Akorri

• NetApp-Onaro • SAN Pulse • TekTools

• Virtual Instruments

z

FC SANs generally require dual fabrics

•

2x the cost

•

Necessity for change management & HA

z

FC 8G is 4G & 2G backwards compatible

(91)

Server Virtualization has Storage

Ramifications

z

Dramatically

increased I/O (storage) demands

z

Patchwork of support, few standards

•

“VMware mode” on storage arrays

•

Virtual HBA/N_Port ID Virtualization (NPIV)

•

Everyone is qualifying everyone and jockeying for position

z

Can be “detrimental” to storage utilization

z

Problematic to traditional BU, replication, reporting

(92)

Virtualized Server Storage Issues

z

Boils down to 4 things to manage

• Performance

• Complexity

• Troubleshooting

(93)

VMware Storage Option-Shared Block Storage

z

Shared storage - common/ workstation approach

•

Stores VMDK image in VMFS datastores

•

DAS or FC/iSCSI SAN

•

Hyper-V VHD is similar

z

Why?

•

Traditional, familiar, common (~90%)

•

Prime features (Storage VMotion, etc)

•

Multipathing, load balancing, failover*

z

But…

•

Overhead of two storage stacks (5-8%)

•

Harder to leverage storage features

•

Often shares storage LUN and queue

•

Difficult storage management

DAS or SAN

VMDKs

(94)

VMware Storage Option-Shared NFS Storage

z

Shared storage on NFS – skip VMFS & use NAS

•

NTFS is the datastore

z

Simple – no SAN

•

Multiple queues

•

Flexible (on-the-fly changes)

•

Simple snap and replicate*

•

Enables full Vmotion

•

Use fixed LACP for trunking

z

But…

•

Doesn’t work w/SRM & storage VMotion

•

CPU load questions

•

Default limited to 8 NFS datastores

•

NAS File limitations

•

Multi-VMDK snap consistency

August 2009 Storage Virtualization School 93 NFS NAS

(95)

VMware Storage Options-Raw Device Mapping

(RDM)

z

Guest VMs access storage directly over iSCSI or FC

•

VMs can even boot from raw devices

•

Hyper-V pass-through LUN is similar

z

Great

•

Per-server queues for performance

•

Easier measurement

•

The only method for clustering

z

But…

•

Tricky VMotion and DRS

•

No storage VMotion

•

More management overhead

•

Limited to 256 LUNs per data center

August 2009 Storage Virtualization School 94 SAN

Mapping File I/O

(96)

Physical vs. Virtual RDM

z Virtual Compatibility Mode

•

Appears same as VMDK on VMFS

•

Retains file locking for clustering

•

Allows VM snapshots, clones,

VMotion

•

Retains same characteristics

If storage is moved

z Physical Compatibility Mode

•

Appears as LUN on a “hard” host

•

Allows V-to-P clustering, VMware

locking

•

No VM snapshots, VCB, VMotion

•

All characteristics & SCSI

commands (except “Report LUN”) are passed through – required for some SAN management software

(97)

Which VMware Storage Method

Performs Best?

Mixed Random I/O

CPU Cost Per I/O

Source: “Performance Characterization of VMFS and RDM Using a SAN”, VMware Inc., 2008

(98)

Server Virtualization Storage Protocol

Breakout per IDC: 2007

August 2009 Storage Virtualization School 97 7% 47% 24% 22% iSCSI SAN FC SAN DAS NAS

(99)

Which Storage Protocol Performs Best?

Throughput by I/O Size

CPU Cost Per I/O

Source: “Comparison of Storage Protocol Performance”, VMware Inc., 2008

(100)

Perplexing Server Virtualization Storage

Performance Problems

z

App performance drop-off

• When moving from physical to virtual servers

• Often causing fruitless guest migrations

• Lots of admin frustration looking for root cause

(101)

The Issue is often…

z

Too Much Oversubscription

z

Generally, oversubscription is a very good thing

(102)

Where Oversubscription Occurs

z

Within the:

• Hypervisor

• LUN

• Disk Drives

• SAN fabric

• Target Storage ports

z

Too much creates positive loop

• Problems feed on themselves

(103)

Hypervisor Oversubscription

z

Hypervisors are designed for oversubscription

•

But too much of a good thing…

Means IO & resource bottlenecks

•

Figuring out the problem root cause is difficult at best

Hypervisor X86 Architecture Ap O p. S p. S Ap O Ap O p. S App. OS

(104)

LUN Oversubscription

z

Combines disks into storage pools

• Each storage pool is carved up by the Hypervisor

Into virtual storage pools

Then assigned to the individual VM guests

Each VM guest contests for the same storage pool

• Storage systems can’t distinguish between guests

Contention decreases traditional Storage performance

Traditional SAN Storage Virtual Storage

(105)

HDD Oversubscription – Especially SATA

z

Slower SATA drives don’t handle contention well

•

Nominal buffers or ques = higher response times

SATA FC/SAS Que depth of 256 to 512 15,000 RPM 10,000 RPM 7,200 RPM Que depth of 0 to 32 Usually 0 7,200 RPM

(106)

SAN Fabric Oversubscription

z

SAN storage is typically oversubscribed

• 8:1 (server initiators – target storage ports) or more

Network blocking can dramatically reduce performance Full storage buffer ques also reduces performance

(107)

Failure to Adjust for Virtual Server

Oversubscription Can be Disastrous

z

SAN or storage target ports blocks IO

•

Causing SCSI timeouts

SCSI drivers are notoriously impatient

Apps crash

Physical oversubscription 8:1

Virtual oversubscription 160:1 •Based on avg 20 guests per VM

(108)

Too Much Oversubscription App Pain Points

Operationally

z

App SCSI timeouts

• Lots of unscheduled downtime

z

Slow app performance

• Reducing productivity

z

Difficult to diagnose causality

• Increased downtime

• Increased user frustration • Increased lost productivity

Economically

z

Too much

• Admin time chasing tail • Scheduled downtime • Unscheduled downtime

z

Lost revenue & productivity

(109)

Too Much Oversubscription Work-Arounds

z

Assign 1:1 physical LUNs to virtual LUNs

•

Easiest with iSCSI Storage

z

Run Hypervisor RDM storage

•

Manually assign storage LUNs to each guest

z

Limit or Eliminate use of SATA

z

Reduce SAN oversubscription ratios

•

Upgrade SAN

8G FC w/NPIV 10G iSCSI

z

Use NAS

•

Eliminates Storage oversubscription

(110)

Issues w/Work-Arounds

z

RDM means ltd advanced features

•

Discouraged by Hypervisor vendors

z

Reduced or eliminate SATA drives

•

Increases costs

Although fat SAS drives are cost effective alternative

z

NAS may cause some app performance issues

•

Oversubscription gains potentially wiped out by performance

z

The key is to look at ecosystem holistically

•

Limit overall oversubscription on the whole

(111)

Better Alternative Can be Virtualized Storage

z

Virtualized SAN &/or NAS (GNS or GFS) Storage

z

Can mitigate or eliminate oversubscription issues by

•

Spreading volumes and files

Across multiple systems, spindles, RAID groups

•

Increasing IO & throughput

By aggregating & virtualizing more HDDs, systems, ports, BW

(112)

Of Course, There is the > Probability

of Data Failure Issue

(Previously Discussed)

z

Probability of 1 system going down is low

•

Probability any 1 of many will fail increases rapidly

(P1+ P2+ P3) – (P1* P2* P3₎

• P = probability of a specific RAID group failing

z

Ways to mitigate increased data failure probabilities

•

Self-healing storage elements

•

RAIR

•

More scalable individual storage systems (SAN or NAS)

(113)

Server Virtualization Lack of End-to-end

Visibility Pain

z

Can’t pierce firewall of server virtualization layer

•

Networked storage mgmt only see the storage side

•

Virtual server mgmt only see the guest side

•

And they do not correlate automatically

z

Difficult to pierce firewall of virtualized storage too

(114)

No Perfect Solutions

z

But some pretty good all around ones

•

Akorri – Balance Point

•

EMC – IT Management Solutions

SMARTS® _{ADM & Family, IT Compliance Mgr, Control Center}®

•

TekTools – Profiler for VM

•

Virtual Instruments – Virtual Wisdom

z

Some focused ones

•

VMware – Veeam Monitor

z

SAN Optimization – Data Migration Tools

•

NetApp – Onaro

•

SAN Pulse – SANlogics

(115)

Server Virtualization DP Pain & Issues

z

Local

z

Wide Area

z

Granularity

(116)

Level Setting Definitions

z

HA protects against local hardware failures

z

DR protects against site failures

z

Business Continuity means

•

No business interruptions for data failures or disasters

z

Data protection software protects against

•

Software failures

•

Human error

•

MalWare

z

Granularity determines

•

Amount of data that can be lost - RPO

•

Amount of time it takes to recover - RTO

(117)

HA Requires Redundant Systems

z

100% redundancy can be a tad expensive

• Upfront & ongoing

• For just protecting against hardware faults

(118)

Virtual Server DR Tends to Work Better w /

SAN or Virtual SAN Storage

z

If hardware hosting VMs fails

• VMs can easily be restarted

Boot from SAN

• On a different physical server

(119)

DR with Shared Storage on SAN

z

Virtual guest images live on the SAN Storage

z

Each VM guest is then pointed @

•

Appropriate storage image & restarted

z

Essentially RTO is zero

•

Or instantaneously

z

All guests & data are protected

•

Available through the SAN

(120)

High Cost of Networked Storage HA

z

Requires duplicated Network Storage for HA

• 2x Network Storage hardware costs

• 2x Network Storage software costs

• More than 2x operational costs

More HA systems means much more costs

(121)

Virtualized Storage Can Mitigate Costs

z

Virtualized storage (SAN or NAS)

• Fewer system images to manage

• Fewer software licenses

• Even Capacity based licenses are less costly

Higher scalability means lower lower costs

(122)

Wide Area DR

Requires VM Storage to mirror

over WAN to remote recovery site

Primary

Site FC over WAN_Gateway FC over WAN_Gateway Recovery Site

Native TCP/IP

(123)

Storage Virtualization (SV) Can Mitigate Costs

z

Not all SV can WAN replicate, ones that do mean

•

Centralized control, fewer points of contact for WAN replication

•

Less admin, less bandwidth contention

•

Better performance

•

Lower software license costs

Primary

(124)

Server Virtualization WAN DR Issues

z

FC over IP Gateways are expensive

•

Cisco & Brocade (QLogic less so)

•

Effective Data Throughput

Greatly reduced by packet loss & distance

Limited packet loss mitigation & WAN opt. has little impact

z

NAS NFS usually has performance issues

•

Native TCP/IP replication effective data throughput reduced

By packet loss & latency

•

Duplicate storage, infrastructure, licenses, maint, etc.

(125)

Reducing Wide Area Issues

z

Replication using native TCP/IP or iSCSI

•

Allows TCP optimizers to be utilized

•

Vendors who offer this type of storage replication include

BlueArc, Compellent, DELL/EQL, EMC, Exanet, Fujitsu, HDS (HNAS),

LHN, NetApp, RELDATA

•

Some Network Storage have TCP optimizers built-in

Fujitsu Eternus 8000 TCP Optimizer TCP Optimizer TCP/IP

(126)

Wide Area DR Technology

z

Hypervisor, Storage, OS, or Application based

• Mirroring – Sync and/or Async

• Snapshot Replication – Async

• CDP

(127)

Typical HA-DR VM Storage & Issues

z

Mirroring

z

Hypervisor snapshot replication

z

CDP typically does not work well over the WAN

z

Traditional Backup

(128)

Mirroring – Sync, Async, Semi-Sync

z

Sync replicates on write

•

Requires remote acknowledgement before local write is released

•

RPO and RTO are fine grain

z

Async releases local writes before remote acknowledged

•

RPO and RTO are medium to fine grain

z

Semi-sync replicates snaps or incremental snaps async

•

RPO and RTO are medium to fine grain

Primary

Site Recovery Site

FC over WAN Gateway FC over WAN Gateway Native TCP/IP Local or remote

(129)

Mirroring Shortcomings

z

Sync

•

Cannot prevent the rolling disaster – disasters are synchronized

•

Expensive & performance limited to ~ 100 circuit miles

z

Async

•

Remote data vaults can be inconsistent and non-recoverable

z

Semi-sync

•

Snapshots are typically not crash consistent

(130)

Snapshot – Simpler HA-DR Alternative?

z

No agents on servers or applications

•

Simple to use

•

Medium to fine granularity RPO & RTO

•

Snapshots sent to other site, potentially bi-directional

•

Snap restores = mount the data, point & you’re done

Remote Snapshot can be promoted to a production volume

•

Fast – virtually instantaneous with no BU Windows

•

Centrally administer w/storage

•

In limited cases – deduped

Snaps

(131)

Storage Virtualization Can Again Help

z

SV that is integrated with BU software provides

•

Centralized control, fewer points of contact for WAN replication

•

Less admin, less bandwidth contention

•

Better performance

•

Lower software license costs

BU Agen

t