3PAR TechCircle
HP Dübendorf
14. April 2011
•
Reto Dorigo
Business Unit Manager Storage
•
Serge Bourgnon
3PAR Business Development Manager
•
Peter Mattei
Senior Storage Consultant
•
Peter Reichmuth
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
Agenda
09:00 – 09:15
Hewlett-Packard Schweiz
Begrüssung
Serge Bourgnon
09:15 – 10:15
Hewlett-Packard Schweiz
HP 3PAR Architecture
Peter Reichmuth
Peter Mattei
10:15 – 10:45
Pause
10:45 – 11:45
Hewlett-Packard Schweiz
HP 3PAR Software + Funktionen
Peter Mattei /
Peter Reichmuth
11:45 – 12.15
Hewlett-Packard Schweiz
Live Demo
Peter Mattei /
3PAR background
•
Founded by server engineers
•
Funded by leading infrastructure providers
•
Commercial shipments since 2002
•
Initial Public Offering, November 2007
•
NYSE: PAR
•
Profitable and strong balance sheet
•
Expanding presence in US, Canada,
Europe, Asia, and Africa
© HP Copyright 2011 – Peter Mattei So ftw a re Se rv ic es O nl in e N ea rl in e P2000 X1000 X3000 X9000 P4000 EVA 3PAR P9500 Data Protector Express Storage Essentials Storage Array Software Storage Mirroring Data Protector Business Copy Continuous Access Cluster Extension
SAN Implementation Storage Performance Analysis
Entry Data Migration Installation & Start-up Data Migration
Proactive 24 Critical Service
Proactive Select Backup & Recovery
SupportPlus 24 SAN Assessment
Consulting services (Consolidation, Virtualization, SAN Design)
Data Protection Remote Support
D2D Backup Systems ESL tape libraries VLS virtual library systems EML tape libraries MSL tape libraries RDX, tape drives
& tape autoloaders
The HP Storage Portfolio
4 In fr a st ru ct ur e
ProCurve Wired, Wireless, Data
Center, Security & Management SAN Connection Portfolio FC Switches/DirectorsB, C & H Series
ProCurve Enterprise Switches
Leading the next storage wave
HP Storageworks Portfolio
Block Level Storage File Level Storage Backup/Recovery
Large Enterprise Federal P9000 (XP) X9000 (IBRIX) StoreOnce Cloud / Hosting
Service Providers 3PAR
Corporate P6000 (EVA) Mid Size X3000 (MS WSS) P4000 (LeftHand) Small/Remote Office
© HP Copyright 2011 – Peter Mattei
Architecture for Cloud Services
• Performance and capacity
scalability for multiple apps
• Handle diverse and unpredictable workloads
• Security among tenants • Resilient
• Acceptable service levels with a major component failure
• High utilization with high performance/service levels
• Eliminate capacity reservations • Allow fat to thin volume migrations
without disruption, post processing
• Continual, intelligent re-thinning without disruption
• Fast implementations of low
overhead RAID levels
• Autonomic configuration, including for server clusters
• Autonomic capacity provisioning
• Autonomic data movement
• Autonomic performance
optimization
• Autonomic storage tiering
Autonomic
Management
Thin
Technologies
Multi-Tenant
Clustering
Built-In, Not Bolt-On
3PAR LEADS IN ALL 3 CATEGORIES
• Mesh Active, Cache Coherent
Cluster
• ASIC-based Mixed Workload
• Virtual Private Array Security
• Tier 1 HA, DR
• Failure-Resistant Performance,
QoS
• Reservation-less,
Dedicate-on-Write
• Thin Engine and Thin API-based Reclamation
• ASIC-based Zero Detection • Wide-Striping, sub-Disk RAID • ASIC-based Fast RAID
• Autonomic Groups
• Autonomic capacity provisioning for thin technologies
• Dynamic Optimization • System Tuner, Policy Advisor • Adaptive Optimization
Autonomic
Management
Thin
Technologies
Multi-Tenant
Clustering
© HP Copyright 2011 – Peter Mattei
3PAR Thin Provisioning
Best new technology in the market
Industry leading technology to maximize storage utilization
Automatically optimizes using multiple classes of storage
Workload management and load balancing
Advanced shared memory architecture
Multi-tenancy for service providers and private clouds
HP 3PAR Industry Leadership
3PAR Autonomic Storage Tiering
3PAR Virtual Domains
3PAR Dynamic Optimization
3PAR Full Mesh Architecture
HP 3PAR InServ Storage Servers
F200
F400
T400
T800
Controller Nodes 2 2 – 4 2 – 4 2 – 8
Fibre Channel Host Ports Optional iSCSI Host Ports Built-in Remote Copy Ports
0 – 12 0 – 8 2 0 – 24 0 – 16 2 0 – 48 0 – 16 2 0 – 96 0 – 32 2 GBs Control/Data Cache 8/12 8-16/12-24 8-16/24-48 8-32/24-96 Disk Drives 16 – 192 16 - 384 16 – 640 16 – 1,280 Drive Types 50GB SSD*, 300, 600GB FC and/or 1, 2TB NL 50GB SSD* 300, 600GB FC and/or 1, 2TB NL 50GB SSD* 300, 600GB FC and/or 1, 2TB NL 50GB SSD* 300, 600GB FC and/or 1, 2TB NL Max Capacity 128TB 384TB 400TB 800TB Throughput/ IOPS (from disk)
1,300 (MB/s) 46,800 2,600 (MB/s) 93,600 3,800 (MB/s) 156,000 5,600 (MB/s) 312,000 SPC-1 Benchmark Results 93,050 224,990
© HP Copyright 2011 – Peter Mattei
Array Comparison
Maximum Values
EVA8400
3PAR T800
P9500
Internal Disks
324
1280
2048
Internal Capacity TB
194/324 ¹
800
1226/2040
3Subsystem Capacity TB
324
800
247‘000
FC Host Ports
8
128/32 ²
192
# of LUNs
2048
NA
65280
Cache GB
22
32+96
512
Sequential Performance Disk GB/s
1.57
6.4
>15
Random Performance Disk IOPS
78’000
>300‘000
>350‘000
Internal Bandwidth GB/s
NA
44.8
192
1 600GB FC / 1TB FATA disks
2 optional iSCSI Host Ports
0 5 10 15 20 25 30 0 25,000 50,000 75,000 100,000 125,000 150,000 175,000 200,000 225,000
HP 3PAR Scalable Performance: SPC-1 Comparison
IBM DS5300
Transaction-intensive applications
typically demand response time < 10 ms
SPC-1 IOPS™
Re
sp
o
ns
e T
im
e (
m
s)
IBM DS8300 Turbo HDS USP V / HP XP24000 EMC CLARiiON CX3-40 NetApp FAS3170 3PAR InServ T800Mid Range
High End
HDS AMS 2500 3PAR InServ F400© HP Copyright 2011 – Peter Mattei
Traditional Modular Storage
Traditional Tradeoffs
Legacy vs. HP 3PAR Hardware Architecture
Cost-efficient but scalability and resiliency limited by dual-controller design
Host Connectivity
Switched Backplane
Traditional Monolithic Storage
Scalable and resilient but costly.
Does not meet multi-tenant requirements efficiently
Cache
Disk Connectivity
Distributed Controller
Functions
Cost-effective, scalable and resilient architecture. Meets cloud-computing requirements for efficiency,
multi-tenancy and autonomic management.
HP 3PAR meshed and active
HP 3PAR – Four Simple Building Blocks
F200 and F400
T400 and T800
Controller Nodes
Performance and connectivity building block CPU, Cache and 3PAR ASIC
System Management RAID and Thin Calculations
Node Mid-Plane
Cache Coherent Interconnect 1.6 GB/sec per Node
Completely Passive encased in steel Defines Scalability
Drive Chassis
Capacity Building Block F Chassis 3u 16 Disk T Chassis 4 U 40 Disks
Service Processor
One 1U SVP per system For service and monitoring
© HP Copyright 2011 – Peter Mattei Gen3 ASIC
Mesh Active
Fast RAID 5 / 6
InForm fine-grained OSUtilization
Manageability
Autonomic Policy ManagementSelf-Configuring
Self-Optimizing
Mixed Workload
Zero Detection
Performance
Instrumentation
Self-Healing
Self-Monitoring
HP 3PAR Utility Storage
Thin Provisioning
Virtual
Domains Virtual Lock
System Reporter Virtual Copy Adaptive Optimization Dynamic Optimization Recovery Managers
F-Class - T-Class
Purpose built on native virtualization
HP 3PAR Architectural differentiation
Remote Copy Thin Conversion Thin Persistence 14
Unified Processor and/or Memory
Control Processor & Memory 3PAR ASIC &
Memory disk Heavy throughput workload applied Heavy transaction workload applied
I/O Processing : Traditional Storage
I/O Processing : 3PAR Controller Node
hostshosts
small IOPs wait for large IOPs to be processed
control information and data are pathed and processed separately
Heavy throughput workload sustained Heavy transaction workload sustained Disk interface
= control information (metadata) = data Host interface Host interface disk Disk interface
Multi-tenant performance
© HP Copyright 2011 – Peter Mattei
Spare Disk Drives vs. Distributed Sparing
HP 3PAR High Availability
Traditional Arrays
3PAR InServ
Few-to-one rebuild
hotspots & long rebuild exposure
Spare drive
Many-to-many rebuild
parallel rebuilds in less time
Spare chunklets
Guaranteed Drive Shelf Availability
HP 3PAR High Availability
S he lf S he lf RAID Group RAID Group She lf S he lf Ra id le t G ro up Ra id le t G ro up Ra id le t G ro up
Traditional Arrays
3PAR InServ
Shelf-dependent RAID
Shelf failure means no access to data
Shelf-independent RAID
© HP Copyright 2011 – Peter Mattei
Write Cache Re-Mirroring
HP 3PAR High Availability
Traditional Arrays
3PAR InServ
Traditional Write-Cache Mirroring
Poor performance due to write-thru mode
Persistent Write-Cache Mirroring
• No write-thru mode – consistent performance
• Works with 4 and more nodes
• F400
• T400
• T800
Write-Cache Mirroring off
HP 3PAR virtualization advantage
RAID5 Set RAID1 Set
RAID1
RAID5 Set RAID6 Set
LUN 1 LUN 0 LUN 3 LUN 4 LUN 5
Traditional Controllers
S p a re S p a re LUN 7 LUN 6 LUN 2 0 1 2 3 4 5 6 7 R1 R1 R5 R6 R6 R1 R5 R5• Each RAID level requires dedicated disks
• Dedicated spare disk required • Limited single LUN performance
Traditional Array
3PAR InServ Controllers
0 1 2 3 4 5 6 7
R1 R1 R5 R6 R6 R1 R5 R5
HP 3PAR
• All RAID levels can reside on same disks
• Distributed sparing
• Built-in wide-striping based on Chunklets
© HP Copyright 2011 – Peter Mattei
HP 3PAR F-Class InServ Components
– Controller Nodes (4U)
• Capacity building block
− 4-Disk Drive Magazines
• Add non-disruptively
• Industry leading density
– 16 Slot Drive Chassis (3U)
– Full-mesh Back-plane
• Post-switch architecture
• High performance, tightly coupled
• Completely passive 3 P A R 4 0 U , 1 9 ” C a b in e t o r C u s to m e r P ro v id e d
• Performance and connectivity building block
− Adapter cards
• Add non-disruptively
• Runs independent OS instance
– Service Processor (1U)
• Remote error detection
• Supports diagnostics and maintenance
• Reporting to 3PAR Central
Configuration Options
HP 3PAR F-Class Node
2 built-in FC Disk Ports 2 built-in FC Disk or Host Ports
Slot 1: optional 2 FC Ports for Host , Disk or FC Replication or 2 GbE iSCSI Ports
Slot 0: optional 2 FC Ports for Host , Disk or FC Replication or 2 GbE iSCSI Ports
GigE Management Port GigE IP Replication Port
– One Xeon Quad-Core 2.33GHz CPU
– One 3PAR Gen3 ASIC per node
– 4GB Control & 6GB Data Cache per node – Built-in I/O ports per node
• 10/100/1000 Ethernet port & RS-232
• Gigabit Ethernet port for Remote Copy • 4 x 4Gb/s FC ports
– Optional I/O per node
• Up to 4 more FC or iSCSI ports (mixable) – Preferred slot usage (in order); depending on
customer requirements
• Disk Connections: Slot 0 (ports 1,2), 0, 1
higher backend connectivity and performance
• Host Connections: Slot 0 (ports 3,4), 1, 0
higher front-end connectivity and performance
• RCFC Connections: Slot 1 or 0
Enables FC based Remote Copy (first node pair only)
• iSCSI Connections: Slot 1, 0
© HP Copyright 2011 – Peter Mattei
– Cache per node
•
Control Cache: 4GB (2 x 2048MB
DIMMs)
•
Data Cache: 6 GB (3 x 2048MB
DIMMs)
– SATA : Local boot disk
– Gen3 ASIC
•
Data Movement
•
XOR RAID Processing
•Built-in Thin Provisioning
– I/O per node
•
3 PCI-X buses/ 2 PCI-X slots and one
onboard 4 port FC HBA
F-Class Controller Node
HP 3PAR InSpire Architecture
Controller Node(s
)
SERIALLAN SATA Data Cache Control Cache 4GB 6 GB 2 – Onboard 4 Port FC 1 0 Quad-Core Xeon 2.33 GHz High Speed Data Links Multifunction Controller 22F-Class DC3 Drive Chassis
Drive Chassis or “cage” contains 4 drive bays that
accommodate:
– 4 drive magazines
– Each magazine holds four disks
– Each disk is individually accessible
© HP Copyright 2011 – Peter Mattei
F-Class DC3 Drive Chassis
– Maximum 16 Drives per Drive Chassis
– Must populate 4 drives (a magazine) at a time – 2 x 4Gb interfaces connected to 2 controller nodes
– Can be Daisy Chained to have 32 drives per loop doubling the amount of capacity behind a node pair
Node 0
Node 1
Node 0
Node 1
Non-Daisy Chained
Daisy Chained
– Minimum configuration is 4 Drive Chassis
– Upgrades must Increment at 4 Drive Chassis
– Must deploy 4 Drive Magazines at a time (16 drives)
across all 4 Drive Chassis (1 drive magazine per Chassis)
*Drive Magazine = 4 disks
Connectivity Options: Per F-Class Node Pair
Ports 0 – 1 Ports 2 - 3 PCI Slot 1 PCI Slot 2 # of FC Host Ports # of iSCSI Ports # of Remote Copy FC Ports # of Drive Chassis Max # of Disks Disk Host - - 4 - - 4 64Disk Host Host - 8 - - 4 64
Disk Host Host Host 12 - - 4 64
Disk Host Host iSCSI 8 4 - 4 64
Disk Host iSCSI RCFC 4 4 2 4 64
Disk Host Disk - 4 - - 8 128
Disk Host Disk Host 8 - - 8 128
Disk Host Disk iSCSI 4 4 - 8 128
Disk Host Disk RCFC 4 - 2 8 128
© HP Copyright 2011 – Peter Mattei
HP 3PAR T-Class InServ Components
• Performance and connectivity building block
− Adapter cards
• Add non-disruptively
• Runs independent OS instance
– Controller Nodes (4U)
• Capacity building block
− Drive Magazines
• Add non-disruptively
• Industry leading density
– Drive Chassis (4U)
– Full-mesh Back-plane
• Post-switch architecture
• High performance, tightly coupled
• Completely passive 3 P A R 4 0 U , 1 9 ” C abi n e t B ui lt -I n C abl e M an ag e me n t
– Service Processor (1U)
• Post-switch architecture
• High performance, tightly coupled
• Completely passive
Bus to Switch to Full Mesh Progression
The 3PAR Evolution
•
3PAR InServ Full Mesh Backplane
• High Performance / Low Latency • Passive Circuit Board
• Slots for Controller Nodes
• Links every controller (Full Mesh) • 1.6 GB/s (4 times 4Gb FC) • 28 links (T800)
• Single hop
•
3PAR InServ T800 with 8 Nodes
• 8 ASICS with 44.8 GB/s bandwidth • 16 Intel® Dual-Core processors • 32 GB of control cache
• 96GB total data cache
• 24 I/O buses, totaling 19.2 GB/s of
peak I/O bandwidth
© HP Copyright 2011 – Peter Mattei
•
2 to 8 per System – installed in pairs
•
2 Intel Dual-Core 2.33 GHz
•
16GB Cache
• 4GB Control/12GB Data
•
Gen3 ASIC
• Data Movement, ThP & XOR RAID
Processing
•
Scalable Connectivity per Node
3 PCI-X buses/ 6 PCI-X slots
• Preferred slot usage (in order)
• 2 slots – 8 FC disk ports
• Up to 3 slots – 24 FC Host ports
• 1 slot – 1 FC port used for Remote Copy
(first node pair only)
• Up to 2 slots – 8 1GbE iSCSI Host ports
Controller Node(s
)
HP 3PAR T-Class Controller Node
T-Class Node pair
0 1 2 3 4 5 0 1 2 3 4 5 PCI Slots
Console port C0
Remote Copy Eth port E1 Mgmt Eth port E0
Host FC/iSCSI/RC FC ports
Disk FC ports
T-Class Controller Node
HP 3PAR InSpire architecture
•
Scalable Performance per Node
•
2 to 8 Nodes per System
•
Gen3 ASIC
•
Data Movement
•
XOR RAID Processing
•Built-in Thin Provisioning
•
2 Intel Dual-Core 2.33 GHz
•
Control Processing
•
SATA : Local boot disk
•
Max host-facing adapters
•
Up to 3 (3 FC / 2 iSCSI)
•
Scalable Connectivity Per Node
•
3 PCI-X buses/ 6 PCI-X slots
Controller Node(s
)
© HP Copyright 2011 – Peter Mattei
T-Class DC04 Drive Chassis
•
From 2 to 10 Drive Magazines
•
(1+1) redundant power supplies
•
Redundant dual FC paths
•
Redundant dual switches
•
Each Magazine always holds
4 disks of the same drive type
•
Each Magazines in a Chassis
can have different Drive types.
For example:
•
3 magazines of FC
•
1 magazine of SSD
•
6 magazines of SATA.
Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2
CD ROM 3PAR Service Processor Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 0 1 2 3 4 5 < > .< > . E 0 E 1 C 0 | | |O O OK OK / ! 0 1 2 3 4 5 < > .< > . E 0 E 1 C 0 | | |O O OK OK / ! 2 T B N L 6 0 0 F C 2 T B N L 6 0 0 F C 2 T B N L 6 0 0 F C 2 T B N L 6 0 0 F C
T400 Configuration examples
– A T400 minimum configuration is
– 2 nodes
– 4 drive chassis with
– 2 magazines per chassis.
– Upgrades are done as columns of
magazines down the drive chassis..
6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C
© HP Copyright 2011 – Peter Mattei
T800 Fully Configured – 224’000 SPC IOPS
• 8 Nodes
• 32 Drive Chassis
• 1280 Drives
• 768TB raw capacity
with 600GB drives
• 224’000 SPC IOPS
Nodes and Chassis are FC connected and can be up to 100 meters apart
Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2
CD ROM 3PAR Service Processor
0 1 2 3 4 5 < >….< >…. E 0 E 1 C 0 | | |O O OK OK / ! 0 1 2 3 4 5 < > ….< >…. E 0 E 1 C 0 | | |O O OK OK / ! 0 1 2 3 4 5 < >….< >…. E 0 E 1 C 0 | | | O O OK OK / ! 0 1 2 3 4 5 < >….< >…. E 0 E 1 C 0 | | | O O OK OK / ! 0 1 2 3 4 5 < >….< >…. E 0 E 1 C 0 | | |O O OK OK / ! 0 1 2 3 4 5 < > ….….< > E 0 E 1 C 0 | | | O O OK OK / ! 0 1 2 3 4 5 < >….< >…. E 0 E 1 C 0 | | |O O OK OK / ! 0 1 2 3 4 5 < > ….….< > E 0 E 1 C 0 | | | O O OK OK / ! 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C 6 0 0 F C Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 Pulizzi | | |0 | | |0 OffOn CB1 OffOn CB2 32
T-Class redundant power
Controller Nodes and Disk Chassis (shelves) are powered by (1+1) redundant power supplies. The Controller Nodes are backed up by a string of two batteries.
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
HP 3PAR InForm OS™
Virtualization Concepts
3PAR Mid-Plane
Example: 4-Node T-Class with 8 Drive Chassis
HP 3PAR Virtualization Concept
•
Drive Chassis are point-to-point
connected to controllers nodes in
the T-Class to provide “cage
level” availability to withstand
the loss of an entire drive
enclosure without losing access to
your data.
•
Nodes are added in pairs for
cache redundancy
•
An InServ with 4 or more nodes
supports “Cache Persistence”
which enables maintenance
windows and upgrades without
performance penalties.
© HP Copyright 2011 – Peter Mattei
Example: 4-Node T-Class with 8 Drive Chassis
HP 3PAR Virtualization Concept
•
T-Class Drive Magazines hold
4 of the very same drives
•
SSD, FC or SATA
•
Size
•
Speed
•
SSD, FC, SATA drive
magazines can be mixed
•
A minimum configuration has 2
magazines per enclosure
•
Each Physical Drive is divided
into 256 MB “Chunklets”
Virtual Volume Virtual Volume
Example: 4-Node T-Class with 8 Drive Chassis
HP 3PAR Virtualization Concept
•
RAID sets will be built across
enclosures and massively striped
to form Logical Disks (LD)
•
LDs are equally allocated to
controller nodes
•
Logical Disks are bound together
to build Virtual Volumes
•
Each Virtual Volume is
automatically wide-striped across
“Chunklets” on all disk spindles
of the same type creating a
massively parallel system
Virtual Volume
Exported LUN
•
Virtual Volumes can now be
exported as LUNs to servers
LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD LD
© HP Copyright 2011 – Peter Mattei
Chunklets – the 3PAR Virtualization Basis
DC
= 256 MB Data Chunklet
SC
= 256 MB Spare Chunklet
DC DC DC DC
Physical Disk
SC
SC
SC
•
Each physical disk in a 3PAR array is
initialized with data and spare
Chunklets of 256MB each
•
Chunklets are Automatically Grouped
by Drive Rotational Speed
Device Type Total # of Chunklets
50GB SSD 185 147GB FC 15K 545 300GB FC 15K 1115 450GB FC 15K 1675 600GB FC 15K 2234 1TB NL 7.2K 3724 2TB NL 7.2K 7225
DC
DC
DC DC
DC
DC
DC
DC DC DC
DC
DC
DC DC DC DC
DC
38Why are Chunklets so Important?
Ease of use and Drive Utilization
•
Same drive spindle can service many different LUNs
and different RAID types at the same time
•
Allows the array to be managed by policy, not by
administrative planning
•
Enables easy mobility between physical disks, RAID
types and service levels by using Dynamic or Adaptive
Optimization
Performance
•
Enables wide-striping across hundreds of disks
•
Avoids hot-spots
•
Allows Data restriping after disk installations
High Availability
•
HA Cage - Protect against a cage (disk tray) failure.
•
HA Magazine - Protect against magazine failure
3PAR InServ Controllers
0 1 2 3 4 5 6 7
R1 R1 R5 R6 R6 R1 R5 R5
© HP Copyright 2011 – Peter Mattei
Common Provisioning Groups (CPG)
CPGs are Policies that define Service and Availability level by
•
Drive type (SSD, FC, SATA)
•
Number of Drives
•
RAID level (R10, R50 2D1P to 8D1P, R60 6D2P or 14D2P)
Multiple CPGs can be configured and optionally overlap the same drives
•
i.e. a System with 200 drives can have one CPG containing all 200 drives and
other CPGs with overlapping subsets of these 200 drives.
CPGs have many functions:
•
They are the policies by which free Chunklets are assembled into logical disks
•
They are a container for existing volumes and used for reporting
•
They are the basis for service levels and our optimization products.
© HP Copyright 2011 – Peter Mattei
Easy and straight forward
Create CPG(s)
– In the “Create CPG” Wizard select
and define
• 3PAR System
• Residing Domain (if any) • Disk Type
− SSD – Solid State Disk − FC – Fibre Channel Disk − NL – Near-Line SATA Disks
• Disk Speed • RAID Type
– By selecting advanced options more
granular options can be defined
• Availability level • Step size
• Preferred Chunklets • Dedicated disks
Easy and straight forward
Create Virtual Volume(s)
– In the “Create Virtual Volume”
Wizard define
• Virtual Volume Name • Size
• Provisioning Type: Fat or Thinly • CPG to be used
• Allocation Warning
• Number of Virtual Volumes
– By selecting advanced options more
options can be defined
• Copy Space Settings • Virtual Volume Geometry
© HP Copyright 2011 – Peter Mattei
Easy and straight forward
Export Virtual Volume(s)
– In the “Export Virtual Volume”
Wizard define
• Host or Host Set to be presented to
– Optionally
• Select specific Array Host Ports • Specify LUN ID
Simplify Provisioning
HP 3PAR Autonomic Groups
Traditional Storage
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 Individual Volumes
Cluster of VMware ESX Servers
Autonomic Host Group
Autonomic Volume Group
– Initial provisioning of the Cluster
• Add hosts to the Host Group
• Add volumes to the Volume Group
• Export Volume Group to the Host Group
– Add another host
• Just add host to the host group
– Add another volume
• Just add the volume to the Volume Group
– Volumes are exported automatically
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10
Autonomic HP 3PAR Storage
– Initial provisioning of the Cluster
• Requires 50 provisioning actions
(1 per host – volume relationship)
– Add another host
• Requires 10 provisioning actions
(1 per volume)
– Add another volume
• Requires 5 provisioning actions
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
HP 3PAR InForm
HP 3PAR Software and Licensing
System Tuner
InForm Operating System
InForm Additional Software
Virtual Copy Thin Persistence Thin Conversion
Thin Provisioning Virtual Domains
Dynamic Optimization
LDAP Virtual Lock
Scheduler Host Personas InForm
Administration Tools
InForm Host Software
Recovery Manager for Oracle
Host Explorer
Recovery Manager for VMware
Multi Path IO IBM AIX
Recovery Manager for Exchange
Multi Path IO Windows 2003
Recover Manager for SQL
System Reporter
3PAR Manager for VMware vCenter
3PAR InForm Software
Thin Copy Reclamation RAID MP (Multi-Parity) Autonomic Groups Rapid Provisioning Access Guard Remote Copy Full Copy Adaptive Optimization
Four License Models:
Consumption Based Spindle Based
Frame Based
Free*
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
HP 3PAR
HP 3PAR Thin Technologies Leadership Overview
Thin Provisioning
– No pool management or
reservations
– No professional services
– Fine capacity allocation units
– Variable QoS for snapshots
Thin Deployments Stay Thin Over time Reduce Tech Refresh
Costs by up to 60% Buy up to 75% less
storage capacity
Start Thin
Get Thin
Stay Thin
Thin Conversion
‣ Eliminate the time & complexity of
getting thin
‣ Open, heterogeneous migrations for
any array to 3PAR
‣ Service levels preserved during inline
conversion
Thin Persistence
‣ Free stranded capacity
‣ Automated reclamation for 3PAR
offered by Symantec, Oracle
‣ Snapshots and Remote Copies stay
© HP Copyright 2011 – Peter Mattei
HP 3PAR Thin Technologies Leadership Overview
•
Built-in
− HP 3PAR Utility Storage is built from the ground up to support Thin
Provisioning (ThP) by eliminating the diminished performance and
functional limitations that plague bolt-on thin solutions.
•
In-band
− Sequences of zeroes are detected by the 3PAR ASIC and not
written to disks. Most other vendors ThP implementation write
zeroes to disks, some can reclaim space as a post-process.
•
Reservation-less
− HP 3PAR ThP draws fine-grained increments from a single free
space reservoir without pre-dedication of any kind. Other vendors
ThP implementation require a separate, pre-dedicated pool for
each data service level.
•
Integrated
− API for direct ThP integration in Symantec File System, VMware,
Oracle ASM and others
Dedicate on write only
HP 3PAR Thin Provisioning – Start Thin
Physically installed Disks
Required net Array Capacities Server presented Capacities / LUNs Physical Disks
Physically installed Disks
Free Chunkl
Traditional Array –
Dedicate on allocation
Dedicate on write only
HP 3PAR Array –
© HP Copyright 2011 – Peter Mattei
HP 3PAR Thin Conversion – Get Thin
Thin your online SAN storage up to 75%
A practical and effective solution to
eliminate costs associated with:
• Storage arrays and capacity
• Software licensing and support • Power, cooling, and floor space
Unique 3PAR Gen3 ASIC with built-in
zero detection delivers:
• Simplicity and speed – eliminate the time &
complexity of getting thin
• Choice - open and heterogeneous migrations for
any-to-3PAR migrations
• Preserved service levels – high performance during
migrations
Before
After
0000 0000 0000 Gen3 ASICFast
52How to get there
HP 3PAR Thin Conversion – Get Thin
1. Defragment source Data
a)
If you are going to do a block level migration via an appliance or host volume
manager (mirroring) you should defragment the filesystem prior to zeroing the free
space
b)
If you are using filesystem copies to do the migration the copy will defragment the
files as it copies eliminating the need to defragment the source filesystem
2. Zero existing volumes via host tools
a)
On Windows use sdelete –c <drive letter> *
b)On UNIX/Linux use dd script
© HP Copyright 2011 – Peter Mattei
HP 3PAR Thin Conversion at a Global Bank
•
No budget for additional storage
•
Recently had huge layoffs
•
Moved 271 TBs, DMX to 3PAR
•
Online/non-disruptive
•
No Professional Services
•
Large capacity savings
•
“The results shown within this
document demonstrate a highly
efficient migration process which
removes the unused storage”
•
“No special host software
components or professional services
are required to utilise this
functionality”
0
50
100
150
200
Unix
ESX
Win
EMC
3PAR
Reduced
power & cooling costs G B sSample volume migrations on different OSs
(VxVM) (VMotion) (SmartMove)
Capacity
requirement
s reduced
by >50%
$3 million
savings
in upfront
capacity
purchases
54Keep your array thin over time
HP 3PAR Thin Persistence – Stay Thin
Before
After
Gen3 ASIC
0000 0000
Fast
– Non-disruptive and
application-transparent “re-thinning” of thin
provisioned volumes
– Thin “insurance” against unexpected
or thin-hostile application behavior
– Returns space to thin provisioned
volumes and to free pool for reuse
– Unique 3PAR Gen3 ASIC with
built-in zero detection delivers:
• Simplicity – No special host software required.
Leverage standard file system tools/scripts to write zero blocks.
• Preserved service levels – zeroes detected and
unmapped at line speeds
– Integrated automated reclamation with
Symantec and Oracle
© HP Copyright 2011 – Peter Mattei
Remember: Deleted files still occupy disk space
HP 3PAR Thin Persistence – manual thin reclaim
LUN 1 Data 1 LUN 2 Data 2 Free Chunklets LUN 1 Data 1 LUN 2 Data 2 Free Chunklets Initial state:
• LUN1 and 2 are ThP Vvols
• Data 1 and 2 is actually written data
LUN 1 Data1 LUN 2 Data 2 Free Chunklets Unused Unused After a while:
• Files deleted by the servers/file system still occupy space on storage
LUN 1 Data1 LUN 2 Data 2 Free Chunklets 00000000 00000000 00000000 00000000
Zero-out unused space:
• Windows: sdelete * • Unix/Linux: dd script
Run Thin Reclamation:
• Compact CPC and Logical Disks
• Freed-up space is returned to the free Chunklets
* sdelete is a free utility available from Microsoft 56
HP 3PAR Thin Persistence and VMware
DataStore
0000000000000000000 0000000000000000000 0000000000000000000
100GB Eager Zeroed Thick VMDK
0 0 0 0 0 0 0 0 0 0
Without 3PAR Thin Persistence
Capacity used = 100GB
All zeroes need tobe written to disk This will impact the performance of the storage
ESX
DataStore
100GB EagerZeroed Thick VMDK
With 3PAR Thin Persistence
Capacity used = 0GB
Hardware zero detectionin the 3PAR Gen3 ASIC No physical disk IO required! ESX 0 0 0 0 0
© HP Copyright 2011 – Peter Mattei
VMware and HP 3PAR Thin Provisioning Options
Storage Array
VMware VMFS
Volume/Datastore
Thin Virtual
Disks (VMDKs)
30GB 150GBVolume
Provisioned at
Storage Array
Virtual Machines (VMs)
Over provisioned VMs: 250 GB 250 GB Physically Allocated: 200 GB 40 GB Capacity Savings: 50GB 210 GB 30GB 150GB 200 GB 200GB Thick LUN 40 GB3PAR Array
10GB 100GB 30GB 150GB 10GB 100GB 30GB 150GB 200GB Thin LUN 10GB 100GB 10GB 100GB 58Built-in not bolt on
HP 3PAR Thin Provisioning positioning
No upfront allocation of storage for Thin Volumes
No performance impact when using Thin Volumes unlike competing
storage products
No restrictions on where 3PAR Thin Volumes should be used unlike
many other storage arrays
Allocation size of 16k which is much smaller than most ThP
implementations
Thin provisioned volumes can be created in under 30 seconds
without any disk layout or configuration planning required
Thin Volumes are autonomically wide striped over all drives within
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
HP 3PAR
HP 3PAR Virtual Copy – Snapshot at its best
Integration with
Oracle, SQL, Exchange, VMware
3PAR Virtual Copy
Base Volume
100s of Snaps…
–
Smart
•
Promotable snapshots
•
Individually deleteable snapshots
•Scheduled creation/deletion
•Consistency groups
–
Thin
•
No reservations needed
•Non-duplicative snapshots
•Thin Provisioning aware
•Variable QoS
–
Ready
•
Instant readable or writeable snapshots
•Snapshots of snapshots
•
Control given to end user for snapshot
management
•
Virtual Lock for retention of read-only snaps
…but just
one CoW
© HP Copyright 2011 – Peter Mattei
HP 3PAR Virtual Copy – Snapshot at its best
–
Base volume and virtual copies can be mapped to different CPG’s
This means that they can have different quality of service
characteristics. For example, the base volume space can be derived
from a RAID 1 CPG on FC disks and the virtual copy space from a
RAID 5 CPG on Nearline disks.
–
The base volume space and the virtual copy space can grow
independently without impacting each other (each space has it’s own
allocation warning and limit).
–
Dynamic optimization can tune the base volume space and the virtual
copy space independently.
HP 3PAR Virtual Copy Relationships
© HP Copyright 2011 – Peter Mattei
Creating a Virtual Copy Using The GUI
Right Click and select “Create Virtual Copy”
InForm GUI View of Virtual Copies
© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Confidentiality label goes here
HP 3PAR
3PAR Remote Copy
HP 3PAR Remote Copy – Protect and share data
– Smart
• Initial setup in minutes
• Simple and intuitive commands • No consulting services
• VMware SRM integration
– Complete
• Native IP-based, or FC
• No extra copies or infrastructure needed • Thin provisioning aware
• Thin conversion
• Synchronous, Asynchronous Periodic or
Synchronous Long Distance (SLD)
• Mirror between any InServ size or model • Many to one, one to many
Sync or Async Perodic Primary Secondary
P
S
S
P
Primary SecondaryP
S
2 TertiaryS
1 Async Periodic Standby SyncSynchronous Long Distance
1:N Configuration
© HP Copyright 2011 – Peter Mattei
2
InServ writes I/Os to secondary cache
Step 2 :
HP 3PAR Remote Copy Synchronous
•
Real-time Mirror
– Highest I/O currency – Lock-step data consistency
•
Space Efficient
– Thin provisioning aware
•
Targeted Use
– Campus-wide business continuity
P
Primary
Volume
Secondary
Volume
S
1
Host server writes I/Os to primary cache
Step 1 :
3
Remote system acknowledges the receipt
of the I/O
Step 3 :
4
I/O complete signal communicated back
to primary host
Step 4 :
Data integrity
HP 3PAR Remote Copy
Assured Data Integrity
– Single Volume
•
All writes to the secondary volume are completed in the
same order as they were written on the primary volume
– Multi-Volume Consistency Group
•
Volumes can be grouped together to maintain write
ordering across the set of volumes
•
Useful for databases or other applications that make
© HP Copyright 2011 – Peter Mattei
The Replication Solution for long-distance implementations
HP 3PAR Remote Copy Asynchronous Periodic
•
Efficient even with high latency replication links
– Host writes are acknowledged as soon as the data is written into cache of the primary array
•
Bandwidth-friendly
– The primary and secondary Volumes are resynchronized periodically either scheduled or manually
– If data is written to the same area of a volume in between resyncs only the last update needs to be resynced
•
Space efficient
– Copy-on-write Snapshot versus full PIT copy – Thin Provisioning-aware
•
Guaranteed Consistency
– Enabled by Volume Groups
– Before a resync starts a snapshot of the Secondary Volume or Volume Group is created
Remote Copy Asynchronous Periodic
Base Volume Snapshot Base Volume Snapshot
Primary Site
P
SequenceRemote Site
A
S
A 1 Initial CopyS
B B-A delta Resynchronization. Delta CopyB
S
A Resynchronization. Starts with snapshots 2Ready for next resynchronization
A
S
AB
S
BUpon Completion. Delete old snapshot 3
© HP Copyright 2011 – Peter Mattei
HP 3PAR Remote Copy many-to-one / one-to-many
•
Asynchronous Periodic Only
•
Distance Limit and Performance
characteristics same as that supported
for asynchronous periodic mode
~4800km /3000 miles and 150ms
•
Requires 2 gigabit Ethernet adapters
per array
•
InServ Requirements
– Max support is 4 to 1.
One of the 4 can mirror bi-directionally
– Requires a minimum of 2 controllers per array per site. Target site requires 4 or more controller
nodes in the array
Primary Site A Primary Site B Primary Site C Primary / Target Site D Target Site P P P P RC RC P RC RC RC 72
Supported Distances and Latencies
HP 3PAR Remote Copy
Remote Copy Type
Max Supported Distance
Max Supported Latency
Synchronous IP
210 km /130 miles
1.3ms
Synchronous FC
210 km /130 miles
1.3ms
Asynchronous Periodic IP
N/A
150ms round trip
Asynchronous Periodic FC
210 km /130 miles
1.3ms
© HP Copyright 2011 – Peter Mattei
Automated ESX Disaster Recovery
VMware ESX DR with SRM
HP 3PAR Servers VMware Infrastructure
Virtual Machines
VirtualCenter Recovery Site Manager HP 3PAR Servers VMware Infrastructure Virtual Machines VirtualCenter Site Recovery Manager
Production Site
Recovery Site• What does it do?
− Simplifies DR and increases reliability
− Integrates VMware Infrastructure with HP 3PAR
Remote Copy and Virtual Copy
− Makes DR protection a property of the VM
− Allowing you to pre-program your disaster
response
− Enables non-disruptive DR testing
• Requirements:
− VMware vSphere™
− VMware vCenter™
− VMware vCenter Site Recovery Manager™
− HP 3PAR Replication Adapter for VMware vCenter
Site Recovery Manager
− HP 3PAR Remote Copy Software
− HP 3PAR Virtual Copy Software (for DR failover
testing)
Production LUNs Remote Copy DR LUNs Virtual Copy Test LUNs 74
HA solution with shared disk resource
Local cluster
Data Center
•
What does it do?
− Provides application failover between servers
•
Advantages:
− No manual intervention required in case of server failure
− Can fail over automatically or manually
•
Disadvantages:
− No protection against storage or Data Center failures
Cluster
© HP Copyright 2011 – Peter Mattei
Using server/volume manager based mirroring
Campus cluster
Cluster
Data Center 1
Data Center 2
A A
•
What does it do?
− Provides very high availability of application/services
− Provides application failover between servers, storage and Data Centers
•
Advantages:
− Data is replicated by OS/volume manager
− No array based replication needed − Storage failure does not require restart
of application/service
− Can fail over automatically or manually
•
Disadvantages:
− High risk for split brain if no arbitration node or service is deployed
− Risk for rolling disaster/data inconsistency