ss481stud

(1)

IBM TotalStorage DS8000

and DS6000 Implementation

Workshop for Open Systems

(Course Code SS48)

Student Notebook

ERC 1.0

cover

(2)

August 2005 Edition

IBM® is a registered trademark of International Business Machines Corporation.

The following are trademarks of International Business Machines Corporation in the United States, or other countries, or both:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product and service names may be trademarks or service marks of others.

AIX DB2 eServer

Enterprise Storage Server ESCON FICON

FlashCopy HACMP Hypervisor

i5/OS iSeries OS/400

POWER POWER5 PowerPC

Predictive Failure Analysis pSeries RS/6000

TotalStorage Virtualization Engine xSeries

(3)

TOC

_Contents

Trademarks . . . xix Course Description . . . xxi Agenda . . . xxiii Unit 1. DS8000 Hardware Components and Architecture . . . 1-1

Unit Objectives . . . 1-3 1.1 DS8000 Highlights . . . 1-5 Topic: DS8000 Highlights . . . 1-6 DS8000 Series Models (2107) . . . 1-7 DS8000 Models . . . 1-8 DS8000 Hardware Overview . . . 1-9 Interfaces to DS8000 (1 of 2) . . . 1-10 Interfaces to DS8000 (2 of 2) . . . 1-12 IBM TotalStorage Management Console . . . 1-13 DS8000 Data Flow . . . 1-14 Host Connectivity: IBM Subsystem Device Driver . . . 1-15 DS8000 Enhancements At a Glance . . . 1-16 DS8000 Overview . . . 1-17 1.2 DS8000 Hardware Components . . . 1-19 Topic: DS8000 Hardware Components . . . 1-20 DS8000: Primary Frame Topology . . . 1-21 DS8000 Terminology . . . 1-22 DS8000 Hardware Components Detail . . . 1-23 DS8000 Processor Complex . . . 1-24 DS8000 Processor Complex: Power5 Server . . . 1-25 DS8000 Processor Complex PC . . . 1-26 DS8000 Processor Complex . . . 1-27 DS8000 Persistent Memory . . . 1-28 DS8000 I/O Enclosure . . . 1-29 RIO-G and I/O Enclosures . . . 1-30 DS8000 RIO-G Port: Layout Example . . . 1-31 Bus Interconnect Performance Comparison . . . 1-32

(4)

DS8000 Switched FC-AL Disk Subsystem . . . .1-44 DS8000 and DS6000 Switched FC-AL/FC-AL . . . .1-45 Switched FC-AL Advantages . . . .1-46 DS8000: Storage Enclosure and DA Cabling . . . .1-47 DS8000 Storage Enclosure and DA Cabling . . . .1-48 1.3 DS8000 Architecture. . . 1-49 Topic: DS8000 Architecture . . . .1-50 DS8000: Primary Frame Topology . . . .1-51 DS8000 Frames . . . .1-52 IBM TotalStorage DS8100 (2-Way) . . . .1-53 DS8300 (4-Way with Two Expansion Frames) . . . .1-54 DS8100 (Model 921) - 2-way . . . .1-55 DS8300 (Models 922 and 9A2) - 4-way . . . .1-56 DS8000 Storage Enclosure Connectivity (1 of 2) . . . .1-57 DS8000 Storage Enclosure Connectivity (2 of 2) . . . .1-58 DS8000 – 2 way - Data Flow . . . .1-59 DS8000 – 4 way - Data Flow . . . .1-60 1.4 DS8000 Cache Management . . . 1-61 Topic: DS8000 Cache Management . . . .1-62 Sequential Prefetching in Adaptive Replacement Cache (SARC) . . . .1-63 DS8000 and DS6000 Caching . . . .1-64 DS8000 and DS6000 Caching Considerations . . . .1-65 1.5 DS8000 RAS Features . . . 1-67 Topic: DS8000 RAS Features . . . .1-68 DS8000 Hardware Components Detail . . . .1-69 Processor Complex RAS . . . .1-70 Hypervisor – Storage Image Independence . . . .1-71 Server RAS (1 of 2) . . . .1-72 Server RAS (2 of 2) . . . .1-73 Server Failover . . . .1-74 NVS Recovery After Complete Power Loss . . . .1-75 Host Connection Availability . . . .1-76 What’s New with SDD 1.6.0.x . . . .1-77 Disk Subsystem . . . .1-78 Disk Subsystem . . . .1-79 Power and Cooling . . . .1-81 Microcode Update . . . .1-82 Management Console . . . .1-83 Unit Summary . . . .1-85

(5)

TOC

DS8000 Switched Loop Layout in an Enclosure Pair . . . 2-9 DS8000 Array Across Loops . . . 2-10 DS6000 Physical Layout . . . 2-11 DS6000 Array Sites on Loop . . . 2-12 DS8000 Creating an Array . . . 2-13 Array Site Configurations: DS8000 Eight DDMs/Array with Arrays Across Loops 2-14 DS8000 AAL Benefits . . . 2-15 Sparing Rules (DS8000) . . . 2-16 Sparing Example 1 - RAID 5: DS8000 All Same Capacity, Same RPM . . . 2-17 Sparing Example 2 - RAID 10: DS8000 All Arrays Same Capacity and Same RPM

. . . 2-18 Sparing Example 3 – DS8000 RAID 5 . . . 2-19 DS6000 Creating an Array . . . 2-20 DS6000: Four DDM Arrays (Not at GA) . . . 2-21 Arrays: Eight DDM Arrays On Loop for DS6000 . . . 2-22 Two Array Sites Configurations: DS6000 Eight DDMs/Array with Arrays on Loop 2-23 One Array Site Configurations: DS6000 (After GA) Four DDMs/Array with Arrays on Loop . . . 2-24 Sparing Rules (DS6000) . . . 2-25 Sparing Example 1 - RAID 5: DS6000 All Same Capacity, Same RPM . . . 2-26 Sparing Example 2 - RAID 10: DS6000 All Same Capacity, Same RPM . . . 2-27 Sparing Example 3 - RAID 5: DS6000 All Same RPM . . . 2-28 Sparing Example 4 – RAID 5: DS6000 Different Size and RPM . . . 2-29 Capacity Magic . . . 2-30 Floating Spare Rules (DS6000 and DS8000) . . . 2-31 DS8000/DS6000: Forming a Rank (Example, FB with 1 GB Extents) . . . 2-33 DS6000/DS8000 Extent Pool Concept Summary . . . 2-34 Device Configuration – Extent . . . 2-35 Extents / Extent Pools Example . . . 2-36 Extent Pools Example in a Mixed CKD and FB Environment . . . 2-37 Logical Volume Concept (1 of 2) . . . 2-38 Logical Volume Concept (2 of 2) . . . 2-39 Virtualization Benefits . . . 2-40 Hosts . . . 2-42 I/O Ports (1 of 2) . . . 2-43 I/O Ports (2 of 2) . . . 2-44 Volume Groups . . . 2-45

(6)

Address Group (2 of 2) . . . .2-56 LSS and Address Group Considerations . . . .2-57 Logical Addressing . . . .2-58 Device Configuration . . . .2-59 Logical Configuration Overview . . . .2-60 Virtualization Hierarchy . . . .2-61 Extent Pools (1 of 4) . . . .2-62 Extent Pools (2 of 4) . . . .2-63 Extent Pools (3 of 4) . . . .2-64 Extent Pools (4 of 4) . . . .2-65 Volumes (1 of 3) . . . .2-66 Volumes (2 of 3) . . . .2-67 Volumes (3 of 3) . . . .2-68 Volume Groups (1 of 4) . . . .2-69 Volume Groups (2 of 4) . . . .2-70 Volume Groups (3 of 4) . . . .2-71 DS6800 LSS Dual Pathed Host (Recommended) . . . .2-72 Volume Groups (4 of 4) . . . .2-73 DS8000 LSS Dual Pathed Host (Recommended) . . . .2-74 Unit Summary . . . .2-76

Unit 3. DS8000 Command Line Interface . . . 3-1

Unit Objectives . . . .3-3 3.1 Introduction and Architecture . . . 3-5 Topic: DS CLI Introduction and Architecture . . . .3-6 DS Command Line Interface (CLI) Introduction . . . .3-7 What’s New? . . . .3-8 Supported DS CLI Platforms . . . .3-9 DSCLI Considerations . . . .3-10 DSCLI Architecture . . . .3-11 Management Consoles: HMC / SMC . . . .3-12 DS CLI Architecture . . . .3-13 DS6000 Data Flow . . . .3-14 3.2 DSCLI Installation . . . 3-15 Topic: DS CLI Installation . . . .3-16 DS CLI Installation (1 of 2) . . . .3-17 DS CLI Installation (2 of 2) . . . .3-18 Unzipped DSCLI CD . . . .3-19 DSCLI - Directory . . . .3-20 Starting DSCLI . . . .3-21

(7)

TOC

DSCLI Interactive Mode . . . 3-30 DSCLI “DSCLI Script” Mode . . . 3-31 3.4 Configuration Process using DSCLI . . . 3-33 Topic: Configuration Process using DSCLI . . . 3-34 DS8000/DS6000 - Storage Virtualization Overview . . . 3-35 Storage Hierarchy Summary . . . 3-36 Physical Infrastructure Hierarchy . . . 3-37 Initial Configuration Prerequisites . . . 3-38 DS CLI User Management . . . 3-39 DSCLI User Groups . . . 3-40 New DSCLI Password Rules . . . 3-41 3.5 DSCLI Commands . . . 3-43 Topic: DSCLI Commands . . . 3-44 DSCLI Command Considerations . . . 3-45 DSCLI Flag Parameters . . . 3-46 Command Flag Parameters or Profile File . . . 3-47 DS CLI Command Types . . . 3-48 List and Show Considerations . . . 3-49 Getting Help with DSCLI . . . 3-50 Object ID Considerations . . . 3-51 Infrastructure IDs . . . 3-52 Object IDs (1 of 5) . . . 3-53 Object IDs (2 of 5) . . . 3-54 Object IDs (3 of 5) . . . 3-55 Object IDs (4 of 5) . . . 3-56 Object IDs (5 of 5) . . . 3-57 1750 Host IOPORT Port Numbers from DS CLI . . . 3-58 Sample CLI Configuration Script (1 of 5) . . . 3-59 Sample CLI Configuration Script (2 of 5) . . . 3-60 Sample CLI Configuration Script (3 of 5) . . . 3-61 Sample CLI Configuration Script (4 of 5) . . . 3-62 Sample CLI Configuration Script (5 of 5) . . . 3-63 Unit Summary . . . 3-65

Unit 4. DS6000 Hardware Architecture . . . 4-1

Unit Objectives . . . 4-3 4.1 DS6000 Highlights . . . 4-5

(8)

DS6800 with One Expansion Enclosure . . . .4-18 DS6000 Code . . . .4-19 Sequential Prefetching in Adaptive Replacement Cache (SARC) . . . .4-20 Disk Enclosure . . . .4-22 Four Data Paths To Every Disk . . . .4-23 Components Architecture (1 of 2) . . . .4-24 Components Architecture (2 of 2) . . . .4-25 DS6000 Host and Device Adapter Ports . . . .4-26 DS6000 Device Adapter Ports for Disk Enclosure EX1 . . . .4-27 DS6000 Switched Disk Expansion (13 Max) . . . .4-28 DS6000 RAID Array Supported Configurations . . . .4-29 Preferred Path – What Is It? . . . .4-30 DS6800 LSS Dual Pathed Host (Recommended) . . . .4-31 DS6800 LSS Single Pathed Host . . . .4-32 4.3 DS6000 Reliability, Availability Serviceability . . . 4-33 Topic: DS6000 Reliability, Availability, Serviceability . . . .4-34 High Availability and Scalability . . . .4-35 Server Failover and Failback . . . .4-37 NVS Recovery After Complete Power Loss . . . .4-38 Major Components – BBUs . . . .4-39 Serviceability . . . .4-40 DS6000 Diagnostics . . . .4-41 Light Path Diagnostics In MR 1750 . . . .4-42 DS6000 Rear Panel . . . .4-43 Rear Panel Indicators . . . .4-44 DS6000 Maintenance and Support: Call Home and Remote Support . . . .4-45 Logs: Realtime . . . .4-46 Log Display: Example . . . .4-47 4.4 DS6000 Installation . . . 4-49 Topic: DS6000 Installation . . . .4-50 DS6000 Installability . . . .4-51 Installation Planning . . . .4-52 Storage Management Console (SMC) . . . .4-54 DS Storage Manager Supported Operating Systems . . . .4-55 DS6000 SMC – Network Topology . . . .4-56 HA with Storage Management Console (SMC) . . . .4-57 Installing Storage and Server Enclosures in a Rack . . . .4-58 DS6000 Server Enclosure Connection Diagram . . . .4-59 Connect Power Cords . . . .4-60 Connect Hosts . . . .4-61

(9)

TOC

Storage Complex Real-Time Functions . . . 4-69 Storage Unit - Real-time Functions . . . 4-70 Assign the Storage Unit . . . 4-71 Storage Unit Properties . . . 4-72 DFSA Feature Activation Web Site . . . 4-73 Update Activation Codes . . . 4-74 Enter Customer Account Information . . . 4-75 Enter Customer Shipping Information . . . 4-76 Enter Customer Contact Information . . . 4-77 Set up Call Home Function . . . 4-78 Update DS6000 Firmware . . . 4-79 Activate Remote Support . . . 4-80 View Performance Reports . . . 4-81 Volume Performance Reports . . . 4-82 Unit Summary . . . 4-84

Unit 5. DS8000 Hardware Management Console . . . 5-1

Unit Objectives . . . 5-2 5.1 DS8000 Storage Management Console . . . 5-3 DS8000 Management Console Overview . . . 5-4 DS8000 Management Console Features . . . 5-5 Hardware Management Console . . . 5-6 DS8000 MC – Network Configuration . . . 5-7 DS8000 MC and a Pair of Ethernet Switches . . . 5-8 DS8000 MC and Ethernet Switches Plugging (Back View) . . . 5-9 DS8000 MC Network Topology . . . 5-10 DS8000 and DS6000 Remote Access Features . . . 5-11 DS8000 MC Network Topology . . . 5-12 DS8000 MC – Network Considerations . . . 5-13 Redundant DS8000 MC . . . 5-14 DS8000 MC – User Management . . . 5-15 DS6000 and DS8000 Remote Support through VPN . . . 5-16 How Virtual Private Network (VPN) Operates . . . 5-17 DS8000 MC – Remote Service Security (1 of 2) . . . 5-18 DS8000 MC – Remote Service Security (2 of 2) . . . 5-19 MC – Network Topology . . . 5-20 DS8000 MC Summary . . . 5-21

(10)

DS Storage Manager Simulated Manager – DS8000 . . . .6-10 DS Storage Manager Realtime and Simulated Managers – DS6000 . . . .6-11 DS Storage Manager and DS CLI . . . .6-12 Realtime versus Simulated Configuration Options . . . .6-13 Exporting Logical Configuration Information . . . .6-14 Unlock Admin Password . . . .6-15 6.2 Installation Requirements . . . 6-17 Topic: Installation Requirements . . . .6-18 DS Storage Manager Supported Operating Systems . . . .6-19 DS Storage Manager Supported Client Browsers . . . .6-20 DS8000 Storage Manager Installation Process (1 of 7) . . . .6-21 DS8000 Storage Manager Installation Process (2 of 7) . . . .6-22 DS8000 Storage Manager Installation Process (3 of 7) . . . .6-23 DS8000 Storage Manager Installation Process (4 of 7) . . . .6-24 DS8000 Storage Manager Installation Process (5 of 7) . . . .6-25 DS8000 Storage Manager Installation Process (6 of 7) . . . .6-26 DS8000 Storage Manager Installation Process (7 of 7) . . . .6-27 6.3 Activating Keys and Storage Unit . . . 6-29 Topic: Activating Keys and Storage Unit . . . .6-30 Storage Image (DS8000*) – Apply Activation Keys . . . .6-31 Storage Image (DS8000*) - Apply Configuration . . . .6-32 Apply Configuration – Select Application Method . . . .6-33 Apply Configuration – Select Storage Complex . . . .6-34 Apply Configuration - Authenticate . . . .6-35 Apply Configuration – Select Storage Unit . . . .6-36 Manage Configuration Files . . . .6-37 User Administration . . . .6-38 Add User . . . .6-39 6.4 Architecture and Logical Configuration Steps . . . 6-41 Topic: Logical Configuration Steps . . . .6-42 Recommended Logical Configuration Steps . . . .6-43 DS Storage Manager Signon . . . .6-44 DS Storage Manager Welcome Panel . . . .6-45 Import Configuration File . . . .6-46 Create Storage Complex (DS8000) . . . .6-47 Create Storage Complex – Define Properties . . . .6-48 Create Storage Unit – General Information . . . .6-49 Create Storage Unit - Specify DDMs . . . .6-50 Create Storage Unit - Define Licensed Function . . . .6-51 Create Storage Unit - Define Licensed Function (Details) . . . .6-52

(11)

TOC

Create Host System – General Information . . . 6-60 Create Host System – Define Host Ports . . . 6-61 Host Attachment Identifier Example . . . 6-62 Create Host System - Specify WWPN . . . 6-63 Create Host System - Specify Storage Image (DS8000*) . . . 6-64 Create Host System – Specify Storage Image Parameters . . . 6-65 Format I/O Ports . . . 6-66 Warning . . . 6-67 Host Attachment Selection . . . 6-68 Create Array . . . 6-69 Create Array – Definition Method . . . 6-70 Create Array - Auto . . . 6-71 Add Arrays to Ranks . . . 6-72 Create Array - Custom . . . 6-73 Create Extent Pool . . . 6-74 Create Extent Pool – Auto (1 of 2) . . . 6-75 Create Extent Pool - Auto (2 of 2) . . . 6-76 Create Extent Pool – Reserve Storage . . . 6-77 Create Extent Pool - Verification . . . 6-78 Create Extent Pool - Custom Define Properties . . . 6-79 Create Extent Pool – Custom Select Ranks . . . 6-80 Create FB Volume . . . 6-81 Create FB Volume – Select Extent Pool . . . 6-82 Create FB Volume – Define Volume Characteristics . . . 6-83 Create FB Volume – Define Volume Properties . . . 6-84 Create FB Volume – Create Volume Nicknames . . . 6-85 Create FB Volume Group . . . 6-86 Create FB Volume Group – Define Properties . . . 6-87 Create FB Volume Group – Select Host Attachments . . . 6-88 Create FB Volume Group – Select Volumes . . . 6-89 Help for Arrays . . . 6-90 DS Storage Manager Summary . . . 6-91 6.5 Summary of Logical Configuration Differences. . . 6-93 Topic: Summary of Logical Configuration Differences . . . 6-94 Summary of Logical Configuration Differences (1 of 2) . . . 6-95 Summary of Logical Configuration Differences (2 of 2) . . . 6-96 DS8000/DS6000 and ESS Differences . . . 6-97

(12)

Storage Image (DS8000*) – Apply Activation Keys . . . .6-109 Storage Image (DS8000*) - Apply Configuration . . . .6-110 Apply Configuration – Select Application Method . . . .6-111 Apply Configuration – Select Storage Complex . . . .6-112 Apply Configuration - Authenticate . . . .6-113 Apply Configuration – Select Storage Unit . . . .6-114 Manage Configuration Files . . . .6-115 User Administration . . . .6-116 Add User . . . .6-117 Help for Arrays . . . .6-118 Unit Summary . . . .6-120

Unit 7. DS8000 and DS6000 Performance, TPC and Productivity Data Center Utility . . . 7-1

Unit Objectives . . . .7-2 7.1 DS8000 Performance . . . 7-3 Topic: DS8000 Performance . . . .7-4 Host Adapter Performance - Single Port . . . .7-5 Maximum Total System IOPS/sec, Open and z/OS . . . .7-6 Sequential Bandwidth . . . .7-7 Total Disk System Sequential Bandwidth . . . .7-8 z/OS Cache Standard Workload . . . .7-9 Open Systems 70/30/50 Workload ESS 800 and DS8100 . . . .7-10 Global Mirror Write Overhead for Transfer . . . .7-11 ESS 800 PPRC versus DS8000 Global Mirror . . . .7-12 FlashCopy Performance Improvements with DS8000 . . . .7-13 7.2 DS6000 Performance . . . 7-15 Topic: DS6000 Performance . . . .7-16 How Migrating to DS6800 Might Affect Hit Ratios . . . .7-17 Cache Read Hit Performance - 512 Byte . . . .7-18 Cache Read Hit Performance - 64 KB . . . .7-19 DS6800 Cache Read Hit Performance (Max IOPs) . . . .7-20 DS6800 Cache Read Hit Response Time . . . .7-21 DS6800 OLTP Workloads (RAID5) . . . .7-22 DS6800 4 KB Read Miss Performance . . . .7-23 Full Box 64 KB Sequential Performance . . . .7-24 Single Stream 64 KB Sequential Performance . . . .7-25 7.3 TotalStorage Productivity Center (TPC) and Productivity Data Center Utility (PDCU) . . . 7-27

(13)

TOC

Configuring and Running PDCU . . . 7-35 Where the PDCU Data is Stored . . . 7-36 PDCU Sample Configuration Data Output . . . 7-37 PDCU Script Invocation . . . 7-39 The Report Generator – Example Macro . . . 7-40 Port Performance Data Collected via PDCU . . . 7-41 Rank Performance Data Collected via PDCU . . . 7-43 Volume Performance Data Collected via PDCU . . . 7-45 Volume Performance Data Collected . . . 7-47 Port Reports . . . 7-49 Rank Reports . . . 7-50 Volume Reports . . . 7-51 PDCU Documentation . . . 7-52 PDCU Summary . . . 7-53 Unit Summary . . . 7-55

Unit 8. DS Copy Services . . . 8-1

Unit Objectives . . . 8-2 8.1 Advanced Copy Services Review . . . 8-3 Topic: Local Flash Copy . . . 8-4 FlashCopy Implementation . . . 8-5 FlashCopy NOCOPY to COPY . . . 8-6 Persistent Flash Copy . . . 8-7 Multiple Relationship FlashCopy . . . 8-8 Incremental FlashCopy . . . 8-9 FlashCopy Consistency Group Support . . . 8-10 8.2 Remote Mirror and Copy . . . 8-11 Topic: Remote Mirror and Copy . . . 8-12 Disk Storage Mirroring . . . 8-13 Metro Mirror (Synchronous PPRC) . . . 8-14 Global Copy (PPRC XD) . . . 8-15 FlashCopy on Remote Mirror and Copy Primary . . . 8-16 IBM TotalStorage Metro Mirror / Global Copy . . . 8-17 PPRC Global Mirror . . . 8-18 PPRC Global Mirror . . . 8-19 PPRC-XD Normal Operations . . . 8-20 PPRC Async Increment Pending . . . 8-21

(14)

Licence for Remote Mirror Copy . . . .8-33 8.4 DSCLI for Copy Services . . . 8-35 Topic: DS CLI for Copy Services . . . .8-36 What’s New? . . . .8-37 ESS Copy Services Command Flow with CS CLI . . . .8-38 DS8000 Copy Services Components . . . .8-39 DS8000 CLI Copy Services Command Flow . . . .8-40 DS6000 CLI Copy Services Command Flow . . . .8-41 CLI Copy Services Coexistence for ESS and DS . . . .8-42 DS CLI Copy Services Commands . . . .8-43 DS CLI FlashCopy . . . .8-44 DS CLI PPRC - Paths . . . .8-45 DS CLI PPRC – Global Mirror . . . .8-46 DS CLI PPRC - Pairs . . . .8-47 DS CLI Clearvol . . . .8-48 DS CLI PPRC . . . .8-49 8.5 DS Storage Manager for Copy Services . . . 8-51 Topic: DS Storage Manager Copy Services GUI . . . .8-52 DS Storage Manager Copy Services GUI . . . .8-53 GUI Create FlashCopy Pairs . . . .8-54 GUI FlashCopy . . . .8-55 GUI Select FlashCopy Options (1 of 2) . . . .8-56 GUI Select FlashCopy Options (2 of 2) . . . .8-57 Paths . . . .8-58 GUI LSS Copy Options . . . .8-59 Metro Mirror Real-time . . . .8-60 DS8000 GUI Copy Services . . . .8-61 Communicate with ESS Copy Services . . . .8-62 Create FlashCopy . . . .8-63 Create a FlashCopy Wizard . . . .8-64 Select a Source Volume . . . .8-65 ESS Copy Services CLI Commands . . . .8-66 DSCLI Copy Services Commands . . . .8-67 Make a FlashCopy . . . .8-68 List Established FlashCopies . . . .8-69 Unit Summary . . . .8-71

Unit 9. DS Host Attachment Consideration . . . 9-1

Unit Objectives . . . .9-2 9.1 AIX Support . . . 9-3

(15)

TOC Remove 2105 Fileset (3 of 9) . . . 9-11 Remove 2105 Fileset (4 of 9) . . . 9-12 Remove 2105 Fileset (5 of 9) . . . 9-13 Remove 2105 Fileset (6 of 9) . . . 9-14 Remove 2105 Fileset (7 of 9) . . . 9-15 Remove 2105 Fileset (8 of 9) . . . 9-16 Remove 2105 Fileset (9 of 9) . . . 9-17 FileSet and MP SDD Installation . . . 9-18 MP SDD Installation (1 of 7) . . . 9-19 MP SDD Installation (2 of 7) . . . 9-20 MP SDD Installation (3 of 7) . . . 9-21 MP SDD Installation (4 of 7) . . . 9-22 MP SDD Installation (5 of 7) . . . 9-23 MP SDD Installation (6 of 7) . . . 9-24 MP SDD Installation (7 of 7) . . . 9-25 Discover WWN of FC HBAs (1 of 3) . . . 9-26 Discover WWN of FC HBAs (2 of 3) . . . 9-27 Discover WWN of FC HBAs (3 of 3) . . . 9-28 Now You Go to the DS6000 (1 of 4) . . . 9-29 Now You Go to the DS6000 (2 of 4) . . . 9-30 Now You Go to the DS6000 (3 of 4) . . . 9-31 Now You Go to the DS6000 (4 of 4) . . . 9-32 Volume Group . . . 9-33 Host Attachment (1 of 3) . . . 9-34 Host Attachment (2 of 3) . . . 9-35 Host Attachment (3 of 3) . . . 9-36 LUN Creation (1 of 2) . . . 9-37 LUN Creation (2 of 2) . . . 9-38 On AIX Server After CFGMGR - No Connection to the Switch . . . 9-39 On AIX Server After GFGMGR - Connection to the Switch . . . 9-40 On AIX Server After CFGMGR - Connection to the Switch and to the DS6000 . . 9-41 After CFGMGR (1 of 3) . . . 9-42 After CFGMGR (2 of 3) . . . 9-43 After CFGMGR (3 of 3) . . . 9-44 After Creation of VG and FS (1 of 3) . . . 9-45 After Creation of VG and FS (2 of 3) . . . 9-46 After Creation of VG and FS (3 of 3) . . . 9-47

(16)

Volume Group on DS6000 (2 of 6) . . . .9-59 Volume Group on DS6000 (3 of 6) . . . .9-60 Volume Group on DS6000 (4 of 6) . . . .9-61 Volume Group on DS6000 (5 of 6) . . . .9-62 Volume Group on DS6000 (6 of 6) . . . .9-63 Host System on DS6000 . . . .9-64 Host System - General Host Information . . . .9-65 Host System - Define Host Ports (1 of 2) . . . .9-66 Host System - Define Host Ports (2 of 2) . . . .9-67 Host System - Define Host WWPN . . . .9-68 Host System - Specify Storage Units . . . .9-69 Host System - Specify Storage Units Parameters . . . .9-70 Host System - Storage Units Parameters . . . .9-71 Host System - Verification . . . .9-72 Host System on DS6000 . . . .9-73 After SDD Install on Windows 2000 (1 of 2) . . . .9-74 After SDD Install on Windows 2000 (2 of 2) . . . .9-75 After Rescan or Reboot (1 of 6) . . . .9-76 After Rescan or Reboot (2 of 6) . . . .9-77 After Rescan or Reboot (3 of 6) . . . .9-78 After Rescan or Reboot (4 of 6) . . . .9-79 After Rescan or Reboot (5 of 6) . . . .9-80 After Rescan or Reboot (6 of 6) . . . .9-81 9.3 Linux Support . . . 9-83 Topics: Linux . . . .9-84 Agenda . . . .9-85 Servers – LINUX RH . . . .9-86 Firmware Checking (1 of 3) . . . .9-87 Firmware Checking (2 of 3) . . . .9-88 Firmware Checking (3 of 3) . . . .9-89 Checking of Number of LUNs (1 of 2) . . . .9-90 Checking of Number of LUNs (2 of 2) . . . .9-91 Rebuild the RAM-Disk (1 of 2) . . . .9-92 Rebuild the RAM-Disk (2 of 2) . . . .9-93 Now You Go to the DS6000 . . . .9-94 Volume Group . . . .9-95 Host Attachment (1 of 2) . . . .9-96 Host Attachment (2 of 2) . . . .9-97 LUN . . . .9-98 MP SDD Installation (1 of 4) . . . .9-99

(17)

TOC

The LUNs Are Not Seen by SDD . . . 9-107 The LUNs Are Seen by SDD . . . 9-108 Add a New Partition to Linux with the LUN from DS6000 (1 of 2) . . . 9-109 Add a New Partition to Linux with the LUN from DS6000 (2 of 2) . . . 9-110 Create a Filesystem (1 of 2) . . . 9-111 Create a Filesystem (2 of 2) . . . 9-112 Mount the Filesystem /PSSCeduc1 (1 of 2) . . . 9-113 Mount the Filesystem /PSSCeduc1 (2 of 2) . . . 9-114 9.4 iSeries Support . . . 9-115 Topics: iSeries . . . 9-116 iSeries Servers – OS/400 and i5/OS (1 of 2) . . . 9-117 iSeries Servers – OS/400 and i5/OS (2 of 2) . . . 9-118 DS6000 Connection to iSeries Systems: Step by Step Commands . . . 9-119 iSeries Host Definition . . . 9-120 iSeries Volume Group Creation . . . 9-121 iSeries Volume Group Creation . . . 9-122 iSeries Volume Group Creation: Summary . . . 9-123 Volume Group Verification . . . 9-124 Host Verification . . . 9-125 Creation of iSeries Protected Volumes on the DS6000 . . . 9-126 DS6000 Volumes View from the iSeries . . . 9-127 Creation of iSeries Unprotected Volumes on the DS6000 . . . 9-128 Volume Group Verification . . . 9-129 DS6000 Volumes Usage in iSeries Disk Pools (1 of 4) . . . 9-130 DS6000 Volumes Usage in iSeries Disk Pools (2 of 4) . . . 9-131 DS6000 Volumes Usage in iSeries Disk Pools (3 of 4) . . . 9-132 DS6000 Volumes Usage in iSeries Disk Pools (4 of 4) . . . 9-133 Unit Summary . . . 9-135

Appendix A. DS8000 LPAR . . . A-1 Appendix B. DS8000 Layout and Cabling Rules . . . B-1 Appendix C. Checkpoint Solutions . . . C-1

(18)

(19)

TMK

_Trademarks

The reader should recognize that the following terms, which appear in the content of this training document, are official trademarks of IBM or other companies:

IBM® is a registered trademark of International Business Machines Corporation.

The following are trademarks of International Business Machines Corporation in the United States, or other countries, or both:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product and service names may be trademarks or service marks of others.

AIX® DB2® eServer™

Enterprise Storage Server® ESCON® FICON®

FlashCopy® HACMP™ Hypervisor™

i5/OS™ iSeries™ OS/400®

POWER™ POWER5™ PowerPC®

Predictive Failure Analysis® pSeries® RS/6000® TotalStorage® Virtualization Engine™ xSeries®

(20)

(21)

pref

_{Course Description}

IBM TotalStorage DS8000 and DS6000 Implementation Workshop for

Open Systems

Duration: 3 days

Purpose

To learn the DS6000/DS8000 storage subsystem and its hardware components and logical structure for Open systems hosts. The

students configure hosts and fixed-block LUNs for the DS8000 storage subsystem using a DS8000 Storage Manager GUI, and review the DSCLI interface for scripting configuration functions. Configuration can be handled online or offline, with import and export capabilities for configuration files created in the offline configuration simulator. This course introduces the DS8000 and its hardware components, and teaches the students the process of configuring the subsystem using the DS8000 GUI or DSCLI interfaces. The DS6000 differences are covered in the class, but most of the labs focus on the DS8000 SM and DSCLI.

Audience

Information Technology (IT) storage administrators responsible for the implementation of storage resources in a SAN environment with hardware solutions an operating systems.

Prerequisites

• SN70 Introduction to Storage Area Networks • SS05 Introduction to Data Storage Systems

• An understanding of zSeries storage and open system operating systems

(22)

DS6000/DS8000

• Describe the DSCLI interface installation and setup

• Describe the DSCLI interface commands and how to use them • Describe the DS Storage Management GUI interface installation

and setup

• Describe the DS Storage Management GUI and how to use the panels to configure the DS6000 subsystem for Open Systems hosts

• Describe the zSeries Host Attachment considerations for the DS6000/DS8000

• Describe the use of the SM GUI and DSCLI for Copy Services • Describe how to access target volumes in Open Systems hosts • Describe the use of scripts and additional programs for managing

Copy Services

• Describe the high-level management interfaces for DS6000 • Describe maintenance and support of the DS6000

• Describe the customer maintenance and CRU replacement procedures

Curriculum relationship

This course is part of the Storage Networking curriculum. A basic understanding of file systems as used in the Open Systems

environment is also a prerequisite. A possible corequisite or follow-on course is SN82 - Planning and Implementing the SAN Volume

Controller. Other courses in this curriculum group are: • SN70 - Introduction to Storage Area Networks • SS05 - Introduction to Data Storage Systems

• SS58 - Planning and Implementing the IBM TotalStorage DS6000 for zSeries

(23)

pref

_Agenda

Day 1

Welcome

Unit 1 - DS8000 Hardware Components and Architecture Unit 2 - DS8000 Virtualization Concepts

Unit 3 - DS8000 Command Line Interface Labs: DSCLI Hands-on Labs

Day 2

Unit 4 - DS6000 Hardware Architecture

Unit 5 - DS8000 Hardware Management Console Unit 6 - DS8000 Storage Manager GUI

Labs: DS Storage Manager GUI Hands-on Labs

Day 3

Unit 7 - DS8000 and DS6000 Performance, TPC and Productivity Data Center Utility

Unit 8 - DS Copy Services

Unit 9 - DS Host Attachment Consideration AIX, Windows, Linux, iSeries

(24)

(25)

Uempty

_{Unit 1. DS8000 Hardware Components and}

Architecture

What This Unit Is About

The DS8000 storage subsystem is the next generation of storage and provides RAID-5 or RAID-10 storage arrays attached to a scalable state-of-the-art processor with variable cache memory and persistent fast-write cache. Power5 1.9 GHz processors process instructions and move data through the subsystem at almost six times the rate of the current ESS storage subsystems. The user can configure the storage subsystem using a GUI-based interface or a DSCLI interface, and configuration can be handled online or offline, with import and export capabilities for configuration files created in the offline configuration simulator. This unit introduces the DS8000 and its hardware

components and architecture. We look at the different models and expansion units, and overview the features and components of the subsystems.

What You Should Be Able to Do

After completing this unit, you should be able to: • Describe the IBM DS8000 highlights

• Describe the IBM DS8000 hardware components • Describe the IBM DS8000 architecture

• Describe the IBM DS8000 Sequential Prefetching in Adaptive Replacement Cache (SARC) management

• Describe the IBM DS8000 RAS features

(26)

SC26-xxxx IBM TotalStorage DS8000 Command Line User's Guide

Http://www.storage.ibm.com/ess http://www.snia.org

Storage Network Industry Association (SNIA) http://www.fibrechannel.org

Fibre Channel Industry Association (FCIA) http://www.t11.org

Technical Committee T11 for device interface of NCITS (National Committee for Information Technology Standards)

(27)

Uempty

Figure 1-1. Unit Objectives SS481.0

Notes:

Unit Objectives

After completing this unit, you should be able to:

Describe the IBM DS8000 highlights

Describe the IBM DS8000 hardware components

Describe the IBM DS8000 architecture

Describe the IBM DS8000 SARC cache management

Describe the IBM DS8000 RAS features

(28)

(29)

(30)

Figure 1-2. Topic: DS8000 Highlights SS481.0

Notes:

Topic: DS8000 Highlights

DS8000 Highlights

DS8000 Hardware Components

DS8000 Architecture

DS8000 Cache Management

DS8000 RAS Features

(31)

Uempty

Figure 1-3. DS8000 Series Models (2107) SS481.0

Notes:

The DS8000 storage subsystem is the next generation of storage, and provides RAID-5 or RAID-10 storage arrays attached to a scalable state-of-the art Power 5 processors with variable cache memory and persistent fast-write cache. Power5 1.9 Ghz processors process instructions, and move data through the subsystem at almost six times the rate of the current ESS storage subsystems. The user can configure the storage subsystem using a GUI-based interface or a CLI interface, and configuration can be handled online or offline, with import and export capabilities for configuration files created in the offline configuration

DS8000 Series Models (2107)

Are:

High Performance

High-capacity series of disk storage

Designed to support continuous operations

Redundancy

Hot replacement / updates

Uses IBM POWER5 server technology

That is integrated with the IBM Virtualization Engine technology

Consist of

Storage unit

One or two (recommended) Management Consoles (MC)

Graphic User Interface (GUI) or Command Line Interface (CLI)

allows

Performing logical configurations and copy services management

functions

(32)

Figure 1-4. DS8000 Models SS481.0

Notes:

There are three models of the DS8000 available to customers. The DS8100 model 921 provides a 2-way 1.5 Ghz Power5 SMP processor and disk storage up to 115 TB. It can address 255 LSSs and 65280 devices, and supports Open Systems LUNs in sizes which can span over multiple arrays, and 3390 devices ranging in size from a single cylinder up to 65520 cylinders in size. It supports fibre channel attachment to SAN environments and supports mainframe PAVs, Multiple Allegiance and I/O priority queuing along with all of the Copy Services functions. The DS8300 provides a 4-way Power5 1.9 Ghz SMP processor and disk storage up to 192 TB.

DS8000 Models

DS8100 (Model 921)

Processor Complex

Dual

Two-way

Up to one expansion frame

DS8300 (Models 922 and 9A2)

Processor Complex

Dual

Four-way

Up to two expansion frames

Model 9A2 supports two IBM TotalStorage System

Logical Partitions (LPAR) in one storage unit

Expansion frame Model 92E attaches to 921 and 922

Expansion frame Model 9AE attaches to 9A2

(33)

Uempty

Figure 1-5. DS8000 Hardware Overview SS481.0

Notes:

The DS8100 provides a 2-way SMP processor and cache memory from 16 to 128 GB. It can house from 2 to 16 FC or ESCON adapters providing 8 to 64 2 GB FCP/FICON ports or 4 to 32 ESCON ports. The base frame, pictured here, can house up to 128 DDMs, and a single expansion frame can add 256 more DDMs for a total of 384 for the DS8100 model 921 and a Model 92E expansion frame. This provides disk storage up to 115 TB. It can address 255 LSSs and 65280 devices and supports 3390 devices up to 65520 cylinders in size. It supports PAVs, Multiple Allegiance and I/O priority queueing along with all the of

Two dual processor servers Up to 128 GB Cache

8 to 64 2 Gb FC/FICON – 4 to 32 ESCON ports 16 to 384 HDD

Intermixable 73 GB 15,000 rpm, 146/300 GB 10,000 rpm Physical capacity from 1.1 TB up to 115 TB

(384 x 300 GB DDMs) 4-Way (Model 8300)

Two four processor servers Up to 256 GB Cache

8 to 128 2 Gb FC/FICON – 4 to 64 ESCON ports 16 to 640 HDD

Intermixable 73 GB 15,000 rpm, 146/300 GB 10,000 rpm Physical capacity from 1.1 TB up to 192 TB

(640 x 300 GB DDMs)

(34)

Figure 1-6. Interfaces to DS8000 (1 of 2) SS481.0

Notes:

The interfaces used to monitor and configure the DS8000 consist of a Web-based GUI and a command-line interface. The DS8000 Storage Manager can be used either in Real-time (online) or Simulated (offline) mode. Either mode is used to manipulate the storage

configuration process for a DS8000, defining CKD or fixed block (FB) storage, or it may be used to modify an existing configuration.

The Simulated Manager which can preconfigure new configurations or modify existing configurations, is to be exported to the DS8000 at a later time. For example, the Simulated Manager could be used to execute or modify changes at an off-peak hour. Normally, a user would access either Real-time or Simulated interfaces through the DS8000, Storage –

Interfaces to DS8000 (1 of 2)

IBM TotalStorage DS Storage Manager (DS-SM: Web based)

Program Interface to perform logical configurations and copy services management functions

Installed via GUI (graphical mode) or as an unattended (silent mode) Accessed through Web browser

Offers:

Simulated Configuration (offline)

Create / Modify / Save logical config when disconnected Apply them to a network attached storage unit

Real-time Configuration (online)

Logical configuration and Copy Services to a network attached storage unit

Both

DS command-line interface (CLI: Script based)

OPEN hosts invoke and manage FlashCopy, Metro and Global Mirror functions

Batch processes and scripts

Check storage unit configuration and perform specific application functions For example:

Check and verify storage unit configuration

Check current copy services config used by storage unit Create new logical storage and copy-services config settings Modify / delete logical storage and copy-services config settings

(35)

Uempty

allows open systems hosts to invoke and manage Copy Services functions as well as to configure and manage all storage units in a storage complex. The DS CLI is a full-function command set.

(36)

Figure 1-7. Interfaces to DS8000 (2 of 2) SS481.0

Notes:

A common Application Programming Interface is also provided for use with storage resource management program such as SRM or TPC.

Interfaces to DS8000 (2 of 2)

DS Open application programming interface (API)

Nonproprietary storage management client application supporting Routine LUN management activities (creation, mapping, masking) Creation / Deletion RAID5 and RAID10 volume spaces

Copy services functions: FlashCopy, PPRC

Helps to integrate configuration management support into existing storage resource management (SRM) applications

Enables automation of configuration management through customer-written applications

Complements the use of Web-based DS-SM and script-based DS-CLI Implemented through IBM TotalStorage Common Information Model (CIM) Agent

Middleware application providing CIM-compliant interface

DS Open API uses CIM technology to manage proprietary devices as open system devices through storage management applications

DS Open API allows these applications to communicate with a storage unit

(37)

Uempty

Figure 1-8. IBM TotalStorage Management Console SS481.0

Notes:

The S-HMC is used to perform configuration, management, and maintenance activities on the DS8000. It can be ordered to be located either physically inside the base frame or externally for mounting in a customer-supplied rack.

If the S-HMC is not operational then it is not possible to perform maintenance, power the DS8000 up or down, or perform Copy Services tasks, such as the establishment of FlashCopies. It is thus recommended to order two management consoles to act as a

IBM TotalStorage Management Console

Focal point for

Configuration, Copy Services, Maintenance

Dedicated workstation installed inside DS8000

Is the eServer Power5 HMC and can be also called S-HMC on

DS8000

Automatic monitoring the state of system

Notify user and IBM when service is required (Call Home)

Can also be connected to network

Enabling centralized

management

through CLI or API scenarios

External management console (optional)

For redundancy with high availability

Called SMC on DS6000

(38)

Figure 1-9. DS8000 Data Flow SS481.0

Notes:

Redundancy in the DS8000 servers allows data to be written into cache and persistent memory of the two servers so that I/Os can be completed and hosts can be released prior to destaging data to disk arrays. This improves write performance and also provides redundancy for failover conditions.

DS8000 Data Flow

The normal flow of data for a

write is the following :

1. Data is written to cache

memory in the owning

server.

2. Data is written to NVS

memory of the alternate

server.

3. The write is reported to the

attached host as having

been completed.

4. The write is destaged from

the cache memory to

disk.

5. The write is then discarded

from the NVS memory of

the alternate server.

Server 0 Server 1 Cache memory for even LSSs NVS for odd LSSs NVS for even LSSs Cache memory for odd LSSs

(39)

Uempty

Figure 1-10. Host Connectivity: IBM Subsystem Device Driver SS481.0

Notes:

For Open Systems hosts a multipath driver is provided for use in load balancing and multipath operation which is fairly common today for disk access. SDD version 1.6.x.x. is provided for the DS6000 and DS8000, and supports most of the common OSs. It can be installed with RDAC for the DS4000 servers as long as each product manages separate HBAs. If third-party multipath drivers or volume managers are used, they supersede the need to install the SDD package.

Host Connectivity:

IBM Subsystem Device Driver

SDD provides the following functions:

Enhanced data availability Automatic path failover

Dynamic I/O load-balancing across multiple paths

Path selection policies for the host system Concurrent download of licensed machine code

With DS6000 and DS8000, SDD is supported on the following OS:

Windows Netware AIX HP-UX SUN Solaris Linux

Can coexist with RDAC (DS4000 multipath driver) on most OS as long they manage separate HBAs Cannot be used with most other multipath drivers (that is, Vertias, PV-Links, Powerpath)

DS6000 Server Device Driver IBM Subsystem Device Driver SCSI

(40)

Figure 1-11. DS8000 Enhancements At a Glance SS481.0

Notes:

Looking at the DS8000 at a glance, there are many reasons why this product is state of the art in disk storage. The new hardware and internal components provide an increase every area of the subsystem making it one of the fastest storage processing subsystems

available. Increased logical addressing provides 16 times the number of available

addresses, and increased connectivity allows connections to many host environments and large SAN installations. The new GUI and CLI interfaces are some of the most robust and functional interfaces in the market, and have adopted a standard look and feel to other IBM TotalStorage products. Major enhancements in the volume administration from the ESS make the DS8000 more flexible and easier to configure than any other storage subsystem. All of these things improve the Total Cost of Ownership or TCO to the customer.

DS8000 Enhancements At a Glance

Hardware – new everything

Processors, adapters, internal paths, frames…

Increased management flexibility via storage system LPARS

Enhanced Performance

Faster or more of almost everything

New patent pending cache algorithms Extended logical device addressing

Up to 256 logical subsystems (LSS) with virtualized assignment of physical capacity to LSSs

Up to 65,280 logical volumes

Extended Connectivity

Up to 128 host ports (FC or Ficon)

Or 64 ESCON host ports

Up to 510 FCP logins per port and 8,192 per storage LPAR

Up to 512 FICON logical paths per logical control unit image and 128,000 per storage facility image

Up to 256 FICON logical path groups per control unit image

Up to 2,048 FICON logical paths per port

Improved volume management

Nondisruptive volume add and delete Up to 65,280 volumes assigned to up to 256 Logical Subsystems (LSSs); an array can contain volumes for multiple LSSs

Larger LUNs (over 2 TB)

64,000 cylinder (55.6 GB) zSeries volumes

Improved Administration

Online and offline configuration capability using a Web-based graphical user interface (GUI)

Ease-of-use improvements (compared to the ESS Specialist)

Command-line interface (CLI) supports control of copy services without

dependencies on GUI-created tasks

Even more attractive Total Cost of Ownership

More flexible feature licensing Four-year standard warranty period Larger capacity volumes supported Increased opportunities for consolidation

(41)

Uempty

Figure 1-12. DS8000 Overview SS481.0

Notes:

Here is a chart which maps the features of the DS8000 according to the various models that are available.

2-way (Base Frame Only) 2-way + Expansion Frame 4-way or LPAR (Base Frame Only) 4-way or LPAR + Expansion Frame 4-way or LPAR + Two Expansion Frames Processor

-pSeries Squadron Power5

2-way 1.5 GHz 2-way 1.5 GHz 4-way 1.9 GHz 4-way 1.9 GHz 4-way 1.9 GHz Cache 16 to 128 GB 16 to 128 GB 32 to 256 GB 32 to 256 GB 32 to 256 GB

Expansion Rack Yes (1) -- Yes (1 or 2) --

--Host Adapters

- per HA: 4-port FC / FICON (2 Gb) - per HA: 2-port ESCON

2 to 16 (example, 8 to 64 FC / FICON ports)

2 to 16 2 to 16 2 to 32 2 to 32

Device Adapters 2 to 8 (example,

1-4 FCALs) 2 to 8 2 to 8 2 to 16 2 to 16 Drives -73 GB (15 K rpm) - 146 GB (10 K rpm) - 300 GB (10 K rpm) 16 to 128 (Increments of 16) 16 to 384 (Increments of 16) 16 to 128 (Increments of 16) 16 to 384 (Increments of 16) 16 to 640 (Increments of 16) Physical Capacity 1.1to 38.4 TB 1.1 to 115 TB 1.1 to 38.4 TB 1.1 to 115 TB 1.1 to 192 TB

Power Three-Phase Three-Phase Three-Phase Three-Phase Three-Phase

Dimensions

-Height x Width x Depth - Footprint 76 x 33.25 x 43 in 9.93 sq. ft. 76 x 66.5 x 43 in 19.86 sq. ft. 76 x 33.25 x 43 in 9.93 sq.ft. 76 x 66.5 x 43 in 19.86 sq. ft. 76 x 99.75 x 43 in 29.79 sq. ft.

DS8000 Overview

(42)

(43)

(44)

Figure 1-13. Topic: DS8000 Hardware Components SS481.0

Notes:

Topic: DS8000 Hardware Components

DS8000 Highlights

DS8000 Hardware Components

DS8000 Architecture

DS8000 Cache Management

DS8000 RAS Features

(45)

Uempty

Figure 1-14. DS8000: Primary Frame Topology SS481.0

Notes:

Here are the front view and the back view of the DS8000 Model 921 or 922 primary frame or base frame. The frame houses the processors, power supplies, host and device

adapters. It also provides a 19-inch rack for up to eight disk enclosures or up to 128 DDMs. Additional DDMs can be added via expansion frames as well as additional adapters. The processor complex is a pair of Power5 p570 servers, and can be 2-way or 4-way

processors. One of the primary advantages of the p5 is the LPAR capability using the 4-way processor. The DS8000 can be split into two distinct logical partitions. One of the

DS8000: Primary Frame Topology

Front Rear

Dense HDD Packaging

16 drives per pack

Dual FC-AL Loop Switches

Point to Point Isolation Two Simultaneous Operations per loop

Storage Hardware Maintenance Console

Processor Complex IBM eServer p5 570

Dual 2-way or Dual 4-way

4 I/O Enclosure Bays

Each bay supports

4 Host Adapters and 2 Device Adapters

Standard 19in rack mounting space Redundant Power BBU: Battery Backup Units Host Adapter 4 FCP/FICON Ports or 2 ESCON Ports Device Adapter 4 FC-AL Ports

(46)

Figure 1-15. DS8000 Terminology SS481.0

Notes:

The group of DS8000 Storage servers it manages is called a Storage Complex and each frame is called a Storage Unit. Each Storage Unit has two pSeries p5 570 servers, and this is called the processor complex. When using the Storage Unit in LPAR mode, the

resources of the Storage Unit can be divided into two logical partitions called LPARs. A Storage Server is the software that is used to control the LPAR. The two LPARs create a Storage Image and each Storage Server has a Storage Image ID associated with it.

DS8000 Terminology

Storage Complex

A group of DS8000s managed by a single Management Console

Storage Unit

A single DS8000 including expansion frames Processor Complex

One P5-570 pSeries server

Two processor complexes form a redundant pair

Divided in one LPAR (models 921 or 922) or two LPARs (model 9A2)

Storage Server

The Software that uses an LPAR

Has access to a percentage of resources available on the Processor Complex for the LPAR

At GA, this percentage is 50% (model 9A2) or 100 % (models 921 or 922) Storage Facility Image (SFI)

Union of two LPARs, one from each Processor Complex

Each LPAR hosts one Storage Server

Processor complex 0 Processor complex 1 Storage facility image 1

Server 0 Storage _facility

image 2

Server 0 Server 1

Server 1

LPARs LPARs

(47)

Uempty

Figure 1-16. DS8000 Hardware Components Detail SS481.0

Notes:

The RIO-G is the backbone of the DS and connects the storage servers to the hosts and to the disk enclosures via device adapter pairs. DA pairs are used from two different RIO-G bays to form a loop between the servers and the device adapters which then connect to the disk enclosures.

DS8000 Hardware Components Detail

HA DA PowerPC Fibre Channel Protocol Proc Fibre Channel Protocol Proc RIO-G Module RIO-G Module PowerPC Fibre Channel Protocol Proc Fibre Channel Protocol Proc RIO-G Interconnect I/O enclosure 4 HAs 2 DAs

2 Gbps Fibre Channel ports 2 Gbps Fibre Channel ports

RIO-G Module POWER5 2-way SMP Processor Memory Processor L1,2 Memory L3 Memory L1,2 Memory POWER5 2-way SMP Memory L3 Memory RIO-G Module Processor Processor L1,2 Memory L1,2 Memory Server 1 Server 0 Processor Complex Processor Complex

(48)

Figure 1-17. DS8000 Processor Complex SS481.0

Notes:

The processor complex is connected to the storage using a high-speed I/O bay called the RIO-G. It contains host adapters and device adapters.

DS8000 Processor Complex

(49)

Uempty

Figure 1-18. DS8000 Processor Complex: Power5 Server SS481.0

Notes:

For storage processing we can get by with 2-way and 4-way. In today’s data processing environments there is no need to go further because nobody requires that kind of speed, however, as the need arises in the future, IBM is well positioned to provide a processor to meet the requirements.

DS8000 Processor Complex : Power5 Server

Processor Adapter Adapter Adapter Adapter To host servers S to rag e ser ver RIO-G Module POWER5 2-way SMP Processor Processor L1,2 Memory L1,2 Memory Memory L3 Memory Processor

The CEC enclosures in the Model 921 each have one processor card (2-way) The CEC enclosures in the Model 922 and 9A2 each have two processor cards

(4-Way)

CEC: Computer Electronic Complex

The CEC Enclosures contain components such as the processor cards, cache memory, and the CEC hard drives

(50)

Figure 1-19. DS8000 Processor Complex PC SS481.0

Notes:

The Power5 processors in the Model 921 are a 2-way 1.5 Ghz compared to a 4-way 600Mhz in an ESS-800. The 922 uses a 1.9 Ghz 4-way processor and provides 6x performance of an ESS-800

The 4-way processor in a 922 can be split at the factory and divided into two LPARs, creating a Model 9A2 and physically splitting the resources of the DS8300 into two logical partitions. Each partition has access to a pair of 2-way processors, two of the RIO-G bays and half the host and device adapters. The disk enclosures are divided between the LPARs. As far as we know this LPAR split is only done at the factory, and is not available as a field upgrade yet.

DS8000 Processor Complex PC

IBM eServer pSeries POWER5 Servers

2-way 1.5 GHz (3X an ESS 800)

4-way 1.9 GHz (6X an ESS 800)

The POWER5 processor supports logical partitioning

The p5 hardware and Hypervisor manage the real-to-virtual

memory mapping to provide robust isolation between LPARs

IBM has been doing LPARs for 20 years in mainframes and 3 years

in pSeries

At GA LPARs are split 50-50, so:

A 4-way has two processors to one LPAR and two processors to

the other LPAR

Post GA, 25-75 possible

LPARs only possible in the 4-way P5s (RIO-G can't be shared in

2-way)

Cache memory from 16 GB-256 GB

Persistent Memory 1 GB to 8 GB dependent on cache size

(51)

Uempty

Figure 1-20. DS8000 Processor Complex SS481.0

Notes:

The Power5 processor is pictured above. The Model 921 has the RIO-G ports on the motherboard, but lacks the second set which are provided in the 922 and 9A2 via a RIO-G card which is installed in one of the PCI-X slots. The processor complex also houses disks for the AIX kernel and storage for backing the persistent memory for fast write cache.

DS8000 Processor Complex

power supply 2

power supply 1

power supply 1 power supply 2

PCI-X slots

RIO-G ports PCI-X adapters with blind swap mechanism

RIO-G ports processor cards

disk drive bays DVD rom drives

operator panel

Front View

(52)

Figure 1-21. DS8000 Persistent Memory SS481.0

Notes:

Persistent memory is the cache memory allocated for fast write operations and, like NVS in the ESS, it is protected from failures and backed by disk and battery power in case of failures in the processor complex or cache. The batteries in the DS8000 hold up the disks in the CEC until the persistent memory can be offloaded to them in the event of a power failure or cache problem.

DS8000 Persistent Memory

The 2107 does not use NVS cards, NVS batteries, or NVS battery

chargers

Data that would have been stored in the 2105 NVS cards resides in

the 2107 CEC cache memory

A part of the system cache is configured to function as NVS

storage

If the 2107 has pinned data in cache, it is written to an extra set of

four disk drives located in each of the CEC enclosures

Six disk drives total in each CEC:

2 for LIC (LVM Mirrored AIX 5.2 + DS8000 code)

4 for pinned data and other CEC functions

During the recovery process the pinned data can be restored from

the extra set of CEC disk drives just like it would have been from the

NVS cards on the ESS 800

(53)

Uempty

Figure 1-22. DS8000 I/O Enclosure SS481.0

Notes:

All base models contain I/O enclosures and adapters. The I/O enclosures hold the adapters and provide connectivity between the adapters and the processors. Device adapters and host adapters are installed in the I/O enclosure. Each I/O enclosure has six slots. Each slot supports PCI-X adapters running at 64 bit, 133 Mhz. Slots 3 and 6 are used for the device adapters. The remaining slots are available to install up to four host adapters per I/O enclosure.

DS8000 I/O Enclosure

(54)

Figure 1-23. RIO-G and I/O Enclosures SS481.0

Notes:

The I/O enclosures hold the adapters and provide connectivity between the adapters and the cache memory over the RIO-G internal bus.

Each adapter has its own PowerPC processor which handles I/O operations independently from the CEC. Host adapters provide FCP connection to attached servers or FICON attached mainframe hosts. Device adapters provide connection to the disk enclosures and the CEC. Internally, the RIO-G bay is connected to the CEC on a high-speed internal bus. SPCN ports access the RIO-G bay for control and monitoring. The device adapter in each RIO-G is one half of a DA pair, and is connected through the disk enclosures to the other DA to form the FC-AL loop.

RIO-G and I/O Enclosures

Also called I/O Drawers

6 PCI-X slots : 3,3 V, 133 MHz blind swap Hot-plug:

Four port Host Adapters cards with four ports each:

FCP or FICON adapters ports

Two Device Adapter cards with four ports each:

4 FC-AL ports per card 2 FC-AL loops per card

Access to cache via RIO-G internal Bus

Each adapter has it's own PowerPC processor Owned by processors in LPAR

SPCN : System Power Control Network

Used to control and monitor the status of the power and cooling within the I/O Enclosure. Cabled as a loop between the different I/O Enclosures

(55)

Uempty

Figure 1-24. DS8000 RIO-G Port: Layout Example SS481.0

Notes:

Each RIO-G port from the CEC connects to the I/O enclosure, and it connects to a second I/O enclosure, and then to the other CEC and back again. The 921 has one loop available, and the 922 and 9A2 have two loops available.

DS8000 RIO-G Port: Layout Example

Up to 4 I/O Enclosures in the same RIO-G loop

Up to 20 I/O Enclosures to the P5-570 system. Processor Complex 0 RIO-G ports Processor Complex 1 RIO-G ports I/O enclosure I/O enclosure

I/O enclosure I/O enclosure

I/O enclosure I/O enclosure I/O enclosure I/O enclosure

Loop 0

Loop 1

Up to four I/O enclosures in the same RIO-G loop Up to 20 I/O enclosures to the P5-570 system.

Each RIO-G port can operate at 1 GHz in bidirectional mode and is capable of passing data in each direction on each cycle of the port. Maximum data rate per I/O enclosure: 4 GB/s.

It is designed as a high-performance self-healing interconnect. The p5-570 provides two external RIO-G ports, and an adapter card adds two more. Two ports on each processor complex form a loop.

Figure shows an illustration of how the RIO-G cabling is laid out in a DS8000 that has eight I/O drawers. This would only occur if an expansion frame were installed. The DS8000 RIO-G cabling varies based on the model.

(56)

Figure 1-25. Bus Interconnect Performance Comparison SS481.0

Notes:

The ESS employed a Common Parts Interconnect bus architecture which was a switch fabric that connected the HA to the processor bays. The ESS has one 500 Mhz link that can provide around 550 MB/sec bandwidth to the servers at maximum throughput. The RIO-G has no CPI switch, but uses 2 133 Mhz 64bit PCI-X buses and 4 -1 Ghz links to the CECs and provides 2000 MB/second per loop. Each 2-way can connect to a RIO-G loop and provide three times the performance and bandwidth of the ESS-800. With a 922 and 4-way processors the DS8300 Model 922 can provide six times the bandwidth of the ESS-800.

Bus Interconnect Performance Comparison

Model 800 CPI/RIO

(4) host bay, (4) host adapters

per bay, (8) CPI buses

CPI bus 32-bit/66 MHz,

connects to EADS

(3) EADS per cluster share (2)

66 MHz/64-bit PHBs PCI

buses

Connects to RIO bus (1 bi-di

link in use per cluster @ 500

MHz 1-byte wide)

Max effective bandwidth: 550

MB/SEC for box (275

MB/SEC bi-di on each RIO

bus in each cluster)

DS8000 RIO-G

(4) I/O Enclosures per RIO-G

loop, (6) host/device adapters

per drawer

No CPI

(2) EADS-X per I/O Enclosure

with (2) 64-bit/133 MHz PHBs

PCI-X buses

Connects to RIO-G bus (4

bi-di links per RIO-G loop @

1 GHz 1-byte wide)

Max effective bandwidth:

2000 MB/SEC per RIO-G

loop, 1 loop per 2-way SMP

(500 MB/SEC bi-di on each of

the 4 RIO-G links)

(57)

Uempty

Figure 1-26. DS8000 Host Adapters HA SS481.0

Notes:

The DS8000 supports two types of host adapters: ESCON and Fibre Channel/FICON. It does not support SCSI adapters.

The ESCON adapter in the DS8000 is a dual ported host adapter for connection to older zSeries hosts that do not support FICON. The ports on the ESCON card use the MT-RJ type connector.

DS8000 Host Adapters HA

(58)

Figure 1-27. Host Adapter with Four Fibre Channel Ports SS481.0

Notes:

Each DS8000 Fibre Channel card offers four 2 Gbps Fibre Channel ports. The cable connector required to attach to this card is an LC type. Each port independently auto-negotiates to either 2 Gbps or 1 Gbps link speed. Each of the four ports on one DS8000 adapter can also independently be either Fibre Channel protocol (FCP) or FICON, though the ports are initially defined as switched point to point FCP.

Each HA has four ports and each port can address 16384 devices so the card can address 4 X 16384 or 65280 devices which is the maximum a DS8000 can address.

The DS8100 has 4 RIO-G bays with 4 HA cards each or 16 HAs, the 922 has 8 RIO-G bays X 4 cards each or 32 HAs.

Host Adapter with Four Fibre Channel Ports

Processor Adapter Adapter Adapter Adapter To host servers S to rag e ser ver Memory PowerPC

2 Gbps Fibre Channel ports

Fibre Channel Protocol Proc HA Fibre Channel Protocol Proc Processor

Configured as FCP or FICON each More FICON logical paths:

ESS (1024) versus DS8000(2048) One FICON channel addresses

16384 devices

One HA card covers all the 65,280 devices that an DS8000 supports Up to 16 HA into a DS8100 or 32

HA into a DS8300

16 FICON channel ports to each single device

Current zSeries Channel subsystems limited to eight channel paths per device Front End of

128 ports for DS8300 (8 times ESS)

64 ports for DS8100 (4 times ESS)

(59)

Uempty

Figure 1-28. DS8000 FCP/FICON Host Adapters: HA SS481.0

Notes:

The definition of the I/O ports is done when the host connection is defined and a WWPN is assigned to the card. The WWPN is not detected from the SAN like it was from the ESS, so these must be entered by hand and errors could occur here.

DS8000 FCP/FICON Host Adapters: HA

Four LC 2 Gb FC ports

Auto-negotiates to 1 Gbps or 2 Gbps

Each port independently auto-negotiates to either 2 Gbps or 1

Gbps link speed.

Can be independently configured to FCP or FICON protocols

The personality of the port is changeable via the DS Storage

Management tools (GUI or CLI).

Ports cannot operate as FCP and FICON simultaneously

FCP port can be long wave or short wave

Short wave ports support a distance of 300 m (non-repeated)

Long wave ports support a distance of 10 km (non-repeated)

(60)

Figure 1-29. DS8000 FICON/FCP Host Adapter SS481.0

Notes:

The card itself is PCI-X 64 Bit 133 MHz. The card is driven by a new high function, high performance ASIC. To ensure maximum data integrity, it supports metadata creation and checking. Each Fibre Channel port supports a maximum of 509 host login IDs. This allows for the creation of very large storage area networks (SANs).

QDR QDR PPC 750GX Flash Buffer Data Protection Data Mover ASIC Protocol Chipset Processor Data Mover 1 GHz PCI-X 64-bit 133 MHz Fibre Channel Protocol Engine Fibre Channel Protocol Engine

Four 2 Gbps Fibre Channel Ports New High Function/High Performance ASIC

Metadata Creation/Checking

Configured at port level Fibre or FICON

SW or LW