Extending Dell Technologies Cloud Platform Availability for Mission Critical Applications

(1)

Extending Dell Technologies Cloud Platform

Availability for Mission Critical Applications

VMware Cloud Foundation on Dell EMC VxRail Multi-Site

Stretched Deployments with Dell EMC PowerMax SRDF/Metro

June 2020

H18367

White Paper

Abstract

This white paper describes a reference architecture used for validating

the use of PowerMax enterprise external storage platforms with

SRDF/Metro in a VMware Cloud Foundation on a Dell EMC VxRail

multisite stretched cluster deployment configuration.

(2)

Copyright

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2020 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA June 2020 White Paper H18367.

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

(3)

Contents

Executive summary

This reference architecture focuses on validating the use of the Dell EMC PowerMax system with SRDF/Metro in a VMware Cloud Foundation on a Dell EMC VxRail multisite stretched-cluster deployment configuration. This configuration provides the highest levels of application availability for customers who are running mission-critical workloads in their Cloud Foundation on VxRail private cloud.

This white paper shows the value that a Dell Technologies portfolio can provide to businesses embarking on their digital transformation.

This white paper is for IT professionals who are interested in deploying a private cloud in their environment and who need validation in leveraging external storage to extend high availability (HA) services for their mission-critical applications.

Dell Technologies and the authors of this document welcome your feedback on the solution and the solution documentation. Contact the Dell Technologies Solutions team by

email or provide your comments by completing our documentation survey.

Authors: Jason Marques, Boon Zhang, Xie Chengcai, Yaodong Zhang, Jing Lily, Krishnamoorthy Karthikeyan

Contributors: Frank Nicolo, Darren Fernandes, Mike Adams, Deepak Vokaliga, Robert Percy, Aighne Kearney

The following table defines abbreviations that used in this document:

Table 1. List of abbreviations

Term Definition

AVN Application Virtual Network AZ Availability Zone

CIDR Classless Inter-Domain Routing HA High Availability

HSRP Hot Standby Router Protocol IQN iSCSI Qualified name LCM Lifecycle Management

SDDC Software Defined Data Center SRDF Symmetrix Remote Data Facility VCF VMware Cloud Foundation VI Virtual infrastructure

vMSC VMware vSphere Metro Storage Cluster

Document purpose Audience We value your Feedback Terminology

(5)

Introduction

Term Definition

vRO vRealize Orchestrator VTEP Virtual Extensible LAN VXLAN VXLAN Tunnel End Point WLD Workload domain

Introduction

Information Technology (IT) departments are under significant pressure to deliver new applications to market, to innovate with technology to surpass competitors, and to do it faster and with more choice. At the same time, requirements exist for stricter compliance, improved security, controlled costs, and increased efficiency. To address these

challenges, the modern data center is trending towards virtualization, converged, hyperconverged, and software-defined infrastructures, and public and hybrid cloud solutions.

The VMware vision of the modern data center is one that is software-defined based on a standardized architecture. The architecture is a fully integrated hardware and software stack that is simple to manage, monitor, and operate. This architecture for the VMware software-defined data center (SDDC) empowers companies to run hybrid clouds and leverage unique capabilities to deliver outcomes that enable efficiency, agility, and

security. The VMware SDDC uses VMware vSphere, VMware vSAN, and VMware NSX to provide compute, storage, and networking virtualization to the SDDC, and the VMware vRealize Suite for additional cloud management, self-service, automation, intelligent operations, and financial transparency.

VMware Cloud Foundation provides integrated cloud infrastructure (vSphere compute, vSAN storage, NSX networking, and security) and cloud management services (with the vRealize Suite) to run many types of enterprise applications in both private and public environments. These applications range from traditional applications that are deployed as virtual machines (VMs) and VMware Horizon virtual desktops to Kubernetes powered containerized cloud-native applications Cloud Foundation helps break down the traditional administrative silos in data centers by merging compute, storage, network provisioning, and cloud management to facilitate end-to-end support for application deployment. The Cloud Foundation SDDC Manager component automates the life cycle management of a complete SDDC on standardized hyperconverged architectures. Customers can deploy Cloud Foundation on-premises on a broad range of supported hardware or consume it as-a-service in the public cloud.

Dell Technologies shares VMware’s vision of the modern data center and extends that vision to the infrastructure. For customers that choose VMware as the primary technology for modernizing their data center or building a multicloud IT environment, Dell

Technologies offer an automated path to the VMware SDDC with VMware Cloud Foundation on VxRail.

The following figure shows the components of the VMware Cloud Foundation on VxRail platform:

(6)

Introduction

Figure 1. Cloud Foundation on VxRail platform components

As the Dell Technologies Cloud Platform, VMware Cloud Foundation on VxRail delivers a simple and direct path to the hybrid cloud and Kubernetes at cloud scale with one

complete automated platform. Cloud Foundation on VxRail builds on native VxRail and Cloud Foundation capabilities, with additional unique integration features that Dell Technologies and VMware have jointly engineered, to help simplify, streamline, and automate the operations of your entire SDDC, from before Day 0 through Day 2 operations. This means you get both the hyperconverged infrastructure and cloud platform software stack in one complete, automated life cycle, turnkey experience. The platform delivers a set of software-defined services for compute, storage, networking, security, cloud management, and end-user computing as well as container-based cloud-native platform services, in both private and public environments. This makes it an ideal operational hub for your hybrid cloud. To accelerate an organization’s move to adopt containers and a hybrid cloud operating model, Dell Technologies offers unique integration between Cloud Foundation and VxRail. The integration supports VM-based and container-based workloads at the same time on Dell EMC PowerEdge servers, with support for optionally incorporating Dell EMC external storage systems across multiple cloud environments.

VMware Cloud Foundation on VxRail makes operating the data center fundamentally simpler by bringing the ease and automation of the public cloud in-house. This is achieved by deploying a standardized and validated network-flexible architecture with integrated full-stack life cycle automation for the entire cloud infrastructure stack, including hardware. This level of deep integration with Cloud Foundation is what gives customers a truly unique turnkey hybrid cloud experience.

(7)

Introduction Further, because security is one of the top priorities for the data center, Cloud Foundation on VxRail includes integrated end-to-end security at every level of the infrastructure -- from compute and management security to microsegmentation at the networking layer, to encryption at the storage layer, to security down into the hardware.

These benefits are also available with single vendor support through Dell Technologies along with choices for infrastructure delivery and flexible financial consumption options. Organizations that are looking to extend their private cloud to the public cloud can consume VMware Cloud Foundation as-a-service through a broad range of cloud service provider partners, including VMware Cloud on AWS, Microsoft Azure, Google Cloud, IBM Cloud, Rackspace, and others. This enables a true hybrid cloud that is based on a

common and compatible platform that stretches from on-premises to off-premises for your VMs and containerized modern applications.

Today, Dell Technologies customers have a lot to consider when choosing the storage that they need to run their workloads. Their storage requirements must scale easily and meet the I/O demands of their applications.

Organizations with large databases, such as those used for big data, machine learning (ML), and AI initiatives, have storage-intensive requirements. Storage must be cost-effective and scale across multiple tiers.

Data protection (backup), disaster recovery, and multisite high availability are other factors in a storage footprint. The flexibility to support different storage types across different platforms is also important. This flexibility helps organizations contribute a consistency that is nondisruptive to their hybrid cloud and multicloud strategy. VMware Cloud Foundation on VxRail offers flexible storage connectivity options that cater to these different use cases to assist customers in their transition to a software-defined platform and operating model.

Customers commonly ask whether they can use external storage in Cloud Foundation on VxRail deployments. The answer is Yes. This helps customers ease into the transition to a software-defined architecture from an operational perspective. It also helps customers leverage the investments in their existing infrastructure for the many different workloads that might still require external storage services.

External storage with Cloud Foundation on VxRail

(8)

Introduction

External storage and Cloud Foundation have two important use cases: principal storage and supplemental storage.

• Principal storage—SDDC Manager provisions a workload domain that uses vSAN, NFS, or Fibre Channel (FC) storage for a workload domain cluster’s principal storage (the initial shared storage that is used to create a cluster.

By default, VCF uses vSAN storage as the principal storage for a cluster. The option to use NFS and FC-connected external storage is also available. This option enables administrators to create a workload domain cluster whose principal storage can be an NFS datastore or an FC-based VMFS datastore instead of vSAN. SDDC Manager provisions a workload domain (WLD) cluster by attaching a previously created NFS file system or FC LUN to it instead of using and configuring a vSAN storage for that cluster. In both cases, the administrator manually creates the external storage ahead of time. The network (IP or FC) must also be configured or zoned ahead of time so that the nodes that will be used for the workload cluster have access to the network. The mounting and formatting of the storage is automated as part of the SDDC Manager ‘create WLD’ cluster workflow.

Importantly, Cloud Foundation does not provide extensible LCM capabilities for the external storage systems. That is still a separately managed process.

• Supplemental storage—This involves manually mounting previously provisioned external NFS, iSCSI, vVols, or FC storage to a Cloud Foundation workload domain cluster that is using vSAN as the principal storage. Supporting external storage for these workload domain clusters is comparable to the experience of administrators using standard vSphere clusters who want to attach secondary datastores to those clusters. For Cloud Foundation, this mounting process is performed manually using standard vCenter operations procedures. Cloud Foundation does not take

ownership of the life cycle management, storage provisioning, or network (IP or FC) configuration or zoning for the external storage systems. Instead, administrators use the native storage system and network management tools for those tasks. At the time of writing, Cloud Foundation on VxRail supports supplemental storage use cases only.

The Dell EMC PowerMax is the first Dell EMC hardware platform that uses an end-to-end Non-Volatile Memory Express (NVMe) architecture for customer data. NVMe is a set of standards that define a PCI Express (PCIe) interface used to efficiently access data storage volumes based on Non-Volatile Memory (NVM) media, which includes modern NAND-based flash along with higher-performing Storage Class Memory (SCM) media technologies. The NVMe-based PowerMax array fully unlocks the bandwidth, IOPS, and latency performance benefits that NVM media and multi-core CPUs offer to host-based applications—benefits that are unattainable using the previous generation of all-flash storage arrays.

The following figure shows the PowerMax 2000 and PowerMax 8000 models:

Use cases

(9)

Introduction

Figure 2. PowerMax product family

The primary benefits that PowerMax platforms offer to customers include: • A powerful end-to-end NVMe storage architecture that delivers:

▪ Up to 15M IOPS, 350 GBps throughput (187K IOPS per rack unit) ▪ Industry-standard NVMe-based flash and SCM drives

▪ Native NVMe Drive Array Enclosures (DAEs)

• The ability to use current Gen 6 FC (FC-NVMe) infrastructures and future-ready Gen 7 FC infrastructures

• Enterprise levels of reliability designed for 99.9999 percent availability in a single array

• Investment protection with the Dell Technologies Future Proof Loyalty Program • Massive workload consolidation by supporting Open Systems (FC, FC-NVMe,

iSCSI), Mainframe, IBM i, containers, and file storage on the same array, simplifying management and significantly lowering the total cost of ownership (TCO)

• Storage provisioning operations in under 30 seconds with Dell EMC Unisphere for PowerMax

• CloudIQ, a simple app to track storage health, report on historical trends, plan for future growth, and proactively discover and remediate issues from any browser or mobile device

• An integrated real-time ML engine for automatic data placement

▪ Automated I/O recognition and data placement across flash and SCM media to maximize performance with no management overhead

(10)

Introduction

▪ Elimination of high-performance silos and consolidation of all mission-critical workloads and secondary applications

• Global inline deduplication and enhanced compression with virtually no performance impact

▪ Data reduction that works with all data services

▪ Granular control that can be turned on or off by application (storage group) • Unprecedented storage security and protection

▪ Controller-based Data at Rest Encryption (D@RE), FIPS 140-2 validated, secure snapshots, role-based authentication, and tamper-proof audit logs • Industry-leading high availability

▪ Non-disruptive hardware and software upgrades ▪ Nondisruptive migrations

PowerMaxOS 5978 and Solutions Enabler/Unisphere for PowerMax 9.1, which were released in Q3 2019, introduced support for SRDF/Metro Online Device Expansion (ODE) and a new Unisphere interface for adding and removing SRDF/Metro devices based on the existing storage group add/remove device workflow. From Unisphere for PowerMax and Solutions Enabler 9.1on, ODE support was expanded to include devices taking part in SRDF/Metro (Active) sessions. This new functionality is based on modifications to our existing Geometry Compatibility Mode (GCM) functionality for host visibility of devices. Unisphere 9.1 also provides new ease-of-use functionality by automating the addition of devices to a storage group, including corresponding SRDF paired devices for single-hop, concurrent, and cascaded SRDF configurations.

With SRDF/Metro, the SRDF secondary device is read/write accessible to the host and takes on the external identity of the primary device (its geometry, device WWN, and so on). By providing this external identity on the secondary device, both the primary and secondary devices appear as a single virtual device across the two SRDF paired arrays for presentation to a single host or host cluster. With both devices accessible, the host (or hosts in the case of a cluster) can read and write to both primary and secondary devices with SRDF/Metro, ensuring that each copy remains current and consistent and addressing any write conflicts that might occur between the paired SRDF devices. A single PowerMax All Flash array can simultaneously support multiple SRDF groups that are configured for SRDF/Metro operations and multiple SRDF groups that are configured for

non-SRDF/Metro operations.

The key differences between SRDF/Metro and standard synchronous and asynchronous SRDF modes are:

• All SRDF device pairs that are in the same SRDF group and are configured for SRDF/Metro must be managed together for all supported operations, with two exceptions:

If all the SRDF device pairs are not ready (NR) on the link, the user may:

▪ Perform a createpair operation to add devices to the SRDF group, provided the new SRDF device pairs are created not ready (NR) on the link.

(11)

Solution architecture ▪ Perform a deletepair operation on all or a subset of the SRDF devices in the

SRDF group.

• An SRDF device pair taking part in an SRDF/Metro configuration can be brought to the following state:

▪ Both sides of the SRDF device pair appear to the host or hosts as the same device.

▪ Both sides of the SRDF device pair are accessible to the host or hosts.

Solution architecture

This section describes the active/active solution design that we implemented by using a Cloud Foundation on a VxRail multisite stretched-cluster deployment configuration with PowerMax SRDF/Metro. This design provides enhanced levels of flexibility and availability that extend the core capabilities of the cloud platform. The VMware Cloud Foundation on VxRail solution natively supports a stretched-cluster configuration for the management domain and a VI workload domain between two availability zones by using vSAN

stretched clusters. A PowerMax SRDF/Metro with vMSC configuration is added to protect VI workload domain workloads by using supplementary storage for the workloads that are running on them.

Two types of vMSC configuration are verified with stretched Cloud Foundation on VxRail: uniform and nonuniform.

• Uniform host access configuration—vSphere hosts from both sites are all connected to a storage node in the storage cluster across all sites. Paths presented to vSphere hosts are stretched across a distance.

• Nonuniform host access configuration—vSphere hosts at each site are connected only to storage nodes at the same site. Paths presented to vSphere hosts from storage nodes are limited to the local site.

The following figure shows the topology of the Cloud Foundation on VxRail uniform stretched-cluster configuration with PowerMax SRDF/Metro.

(12)

Solution architecture

Figure 3. Cloud Foundation on VxRail uniform stretched-cluster configuration with PowerMax SRDF/Metro

The following figure shows the topology of the Cloud Foundation on VxRail nonuniform stretched-cluster configuration with PowerMax SRDF/Metro:

(13)

Figure 4. Cloud Foundation on VxRail nonuniform stretched-cluster configuration with PowerMax SRDF/Metro

In total, we used fourteen VxRail E460F nodes running VxRail version 4.7.410 code on each node, which aligns with Cloud Foundation 3.9.1 requirements. For a complete bill of materials, see VMware Cloud Foundation 3.9.1 on Dell EMC VxRail Release Notes. The following table shows the hardware components that we used in this configuration. For this deployment, we deployed the VMware Cloud Builder utility on the VCF

management domain cluster. If required, you can deploy this utility on separate infrastructure. We used a separate PowerEdge R630 server to host the vSAN witness components for the stretched clusters.

Table 2. Hardware resources

Hardware Quantity Purpose Operating system/Firmware

VxRail E460F server 8 Stretched VCF management domain, VCF Cloud Builder VM, and NSX DHCP VM ESXi6.7 U3b VxRail E460F server 6 Stretched VI workload domain connected to PowerMax SRDF/Metro devices ESXi6.7 U3b Dell EMC PowerEdge R630 server

1 Stand-alone server used as witness host for VCF vSAN stretched cluster

ESXi6.7 U3b

Hardware components

(14)

Hardware Quantity Purpose Operating system/Firmware

PowerMax 8000 Array

1 External storage for VCF VI workload domain. R1 of SRDF configuration

PowerMaxOS 5978

PowerMax 2000 Array

1 External storage for VCF VI workload domain. R2 of SRDF configuration PowerMaxOS 5978 Cisco Nexus 9372PX Switch

2 L3 switches are used for VCF management, vMotion, vSAN, NSX VxLAN, vRealize Network

7.0(3)I5(2)

The following table shows the software components that we used in this configuration:

Table 3. Software resources

Software Version

VMware Cloud Foundation 3.9.1 VMware vSphere ESXi6.7 U3b Dell EMC VxRail 4.7.410 Dell EMC PowerPath/VE 7.0 Dell EMC Unisphere for PowerMax 9.2.0.1194 Dell EMC vRO plug-in for PowerMax 1.2.0.397

For information about firmware and software versions, see Dell EMC VxRail Appliance Release Notes, Version 4.7.x.

The following table shows the infrastructure VMs that we used in this configuration:

Table 4. Infrastructure VMs

Infrastructure VM Purpose

DHCP VM VxLAN - DHCP Network

vCenter Temporary vCenter to add the ESXi server and configure NTP and hostname

VCF Cloud Builder VM Build up the VCF management domain

vSAN Witness Witness VM for stretched VCF cluster from VMware-VirtualSAN-Witness-201912001-15160138.ova Unisphere VM A Windows 2012R2 VM runs the Dell EMC Unisphere

application Dell EMC PowerPath

Management Appliance

Provides license automation to support Dell EMC PowerPath/VE. The PPMA version is 2.6

vCenter Two in the management cluster: the primary vCenter for the management domain and the vCenter for the workload domain

Software components

Infrastructure virtual machines

(15)

Infrastructure VM Purpose

Jumpbox Windows VM provides RDP service to access the VCF cluster

For Ethernet/iSCSI:

• Each node is connected to two 10G uplinks. A pair of Cisco Nexus 9372PX switches is used as top-of-rack (ToR) switches.

• Uplink connectivity to the core network through the ToR switches.

• Intel X520 dual-port 10Gb NIC is used for iSCSI connections to PowerMax. Each VxRail workload domain node has one network daughter card (NDC).

For FC:

• Emulex LPe16002 dual-port 16Gb FC adapter is used for the FC connection to PowerMax. Each VxRail workload domain node has one FC HBA card.

The following figure shows the physical connectivity between the servers, arrays, and switches:

Figure 5. Physical connectivity between servers, arrays, and switches

(16)

Configuration

The building block of this solution uses VMware Cloud Foundation 3.9.1 on VxRail 4.7.410 with PowerMax 8000/2000. The following sections of this document describe the key configuration steps.

Multiple VLANs are designed to isolate traffic across AZ1 and AZ2. All networking is performed at layer 2. No layer 3 routing is required on the network for AZ1 hosts to reach AZ2 hosts.

The following figure shows the network topology:

Figure 6. Network topology

L2 network overview

The management domain network configuration is recorded in the vcf-vxrail-deployment-parameter.xlsx spreadsheet. This configuration is used for deploying the management domain on AZ1 using the VMware CloudBuilder utility.

• The port group names vCenter Server Network, vSphere vMotion, and vSAN are VxRail default names. These names are not editable in the VCF deployment parameter spreadsheet.

• The port group name VXLAN (VTEP) is the default name for NSX-v deployments in VCF 3.9.1. Where “n/a” is displayed for CIDR and gateway, these values are not relevant for the VCF deployment. For vMotion and vSAN, CIDR is set during the VxRail cluster deployment. The VXLAN (VTEP) CIDR is set by DHCP.

The following table shows the management domain network configuration details:

(17)

Configuration

Table 5. Management domain networks

VLAN number

Port group name

CIDR

notation Gateway MTU

2312 vCenter Server Network

10.226.131.6 4/26

10.226.131.65 1500

1602 vSphere vMotion n/a n/a n/a 1601 vSAN n/a n/a n/a 1603 VXLAN (VTEP) - DHCP Network n/a n/a 9000 2313 nsxv-uplink01 10.226.131.1 28/28 10.226.131.129 9000 2314 nsxv-uplink02 10.226.131.1 44/28 10.226.131.145 9000 L2 port setting

Ports for all hosts are configured in the management cluster, as shown in the following figure:

Figure 7. Switch port interface setting

VLAN setting on AZ1 and AZ2

The default gateways for the management network and NSX-v uplink networks are configured on the network core. HSRP is used for redundancy, as shown in the following figure:

(18)

Configuration

• The VxRail ToR switches are connected to the lab core switches. The lab network is entirely layer 2, so all default gateways are configured on the core switches. No default gateways are configured on the VxRail ToR switches because both management and workload domain clusters are stretched using layer 2.

• All VLANs are stretched from AZ1 to AZ2 and subnets. The default gateways are the same.

vSAN Witness

Two vSAN witness appliances are deployed to the stand-alone ESXi host, one each for the management and workload domain clusters. Layer 2 VLANs are stretched to the “third site” to facilitate the layer 2 stretch deployment methodology. vSAN witness appliances are deployed onto the management VLAN with an additional interface on the vSAN VLAN.

Configuring VCF stretched-cluster deployments requires first stretching the management domain cluster and then stretching workload domain clusters, as described in the

following procedure.

Management domain cluster:

1. Stretch VLANs for management, vSAN, and vMotion to AZ2.

2. Use the SoS command within SDDC CLI to prepare the cluster to be stretched. 3. Power on AZ2 management domain hosts and perform cluster expansion in

SDDC Manager.

4. Deploy the vSAN witness appliance to an ESXi host in a third location and add the appliance to the domain being stretched.

5. Use the SoS command in the SDDC CLI to complete the L2 stretch operation. 6. Monitor vSAN health to ensure that the stretch operation is successful. VI workload domain clusters:

• Repeat the preceding steps for each workload domain cluster to be stretched. We used the same process to stretch management and workload domains across two availability zones. To configure layer 2, we followed the process that is described in the VMware document Stretch a Cluster for NSX-V in VMware Cloud Foundation 3.9.1. First deploy the Cloud Foundation stretched management domain cluster on VxRail, and then deploy the Cloud Foundation stretched workload domain cluster on VxRail. After a successful deployment, add the workload domain by using the wizard that is available in the SDDC Manager UI.

The following figure shows an example:

Stretched-cluster configuration in Cloud

(19)

Configuration

Figure 9. SDDC Manager UI: W Dashboard view of stretched VxRail management and VI workload domains

Deploying vRA with embedded vRO in Cloud Foundation involves:

• Deploying the vRealize Suite Lifecycle Manager virtual appliance onto the management domain.

• Deploying vRA using the vRSLCM onto the management domain. • Enabling embedded vRO within the vRA deployment.

The following sections of the document describe these procedures.

Deploy vRealize Suite Lifecycle Manager using SDDC Manager

In the SDDC Manager UI:

1. From the VMware repository, download the vRealize LCM bundle. 2. Create all required DNS records.

3. Use the wizard that is available in SDDC Manager to deploy vRSLCM onto the management domain.

The following figure shows the vRealize Suite Lifecycle Manager UI:

Deploying vRealize

Automation and vRealize

(20)

Configuration

Figure 10. vRealize Suite Lifecycle Manager

Deploy vRealize Automation

In the SDDC Manager UI: 1. Add the vRA license key.

2. Create all the DNS records that are required for vRA components. 3. Download the vRA bundle from the VMware repository.

4. Create a multi-SAN SSL certificate request within the SDDC Manager CLI and sign it using lab certificate authority.

5. Create Microsoft SQL server and configure it for vRA. 6. Create a Windows OVA template for vRA IaaS components.

7. Deploy vRA from within the vRealize Suite menu of the SDDC Manager UI.

Notes:

vRA is deployed to the SDDC management cluster, onto an NSX Application Virtual Network (AVN) which you specify during the Cloud Foundation deployment.

After successful deployment of vRA, add the workload domain by using the wizard that is available in the SDDC Manager UI. The following figure shows an example:

(21)

Configuration

Figure 11. SDDC Manager UI: Deploying vRealize Automation

For more information about the preceding steps, see the VMware document Deploy vRealize Automation in Cloud Foundation.

Enable vRealize Orchestrator

To automatically create SRDF/Metro on PowerMax, you must enable embedded vRO on each vRA node:

1. SSH into vRA node with the root account.

2. Run chkconfig vco-configurator to verify the service.

3. Run chkconfig vco-configurator on to enable vRO at boot on vRA node. 4. Run service vco-configurator status to check the service status. The following figure shows the vRealize Orchestrator UI:

(22)

Configuration

For more information about the process, see the VMware document Start the vRealize Orchestrator Configurator Service in Cloud Foundation.

The PowerMax SRDF/Metro feature enables an administrator to host applications in HA environments. When configured using SRDF/Metro, hosts, clusters, and applications can perform read/write operations on both source and target arrays. The feature functions like SRDF/S mode and maintains a real-time copy at arrays that are located within 200 KMs, depending on application workload, network latency, and block size.

To create SRDF/Metro pairs:

1. Create volumes on both source and target arrays as needed. 2. Add the volumes to storage groups on each side.

3. Create SRDF/Metro pairs between the source and target array (see Create

SRDF/Metro pairs with vRO plug-in for PowerMax).

4. Wait until the synchronization completes and the arrays are in an active/active pair state.

5. Perform storage provisioning tasks on the source and target arrays (see Create

SRDF/Metro pairs with vRO plug-in for PowerMax).

Note: SRDF/Metro offers customers two types of witness deployment models: a physical array

witness and a software witness. We configured this validated SRDF/Metro environment using the physical array witness.

Create SRDF/Metro pairs with vRO plug-in for PowerMax

The vRO plugin for PowerMax enables the administrator to provide automation

capabilities for Dell EMC VMAX All Flash and PowerMax storage arrays within vRO. The plug-in helps automate the operations that are required for provisioning and protection with prescheduled and customized workflows.

1. The first time the vRO plug-in for PowerMax is used, add the Storage Management Server using Dell EMC Unisphere for PowerMax.

Run the “Add Storage Management Server” workflow, and then run the “Register Storage System” workflow, as shown in the following figure:

PowerMax SRDF/Metro

(23)

Configuration

Figure 13. Adding and registering Dell EMC Unisphere for PowerMax in vRO

2. Create the host on the source and target PowerMax, as shown in the following figure. Run the “Create Host” workflow.

For FC:

▪ If the FC zoning is complete, the initiator WWN has already logged into PowerMax. Select the WWN in Pick initiator WWNs from available ones on array. When you click + and enter part of the WWN, the full WWN appears automatically.

(24)

Configuration

Figure 14. Create host for FC

For iSCSI:

(25)

Configuration

Figure 15. Create host for iSCSI

3. Configure auto storage provisioning on the source PowerMax. Run the “Provision Volumes to Host Group” workflow.

In the Host Group tab, enter the host group name and select the hosts that you created in the preceding step. Select Fiber or iSCSI:

▪ For a uniform vMSC configuration, select AZ1 and AZ2 hosts into the host group, as shown in the following figure:

(26)

Configuration

Figure 16. Provision volumes - host group for uniform

▪ For a nonuniform vMSC configuration, select AZ1 hosts into the host group for PowerMax R1 and select AZ2 hosts into the host group for PowerMax R2, as shown in the following figure:

(27)

Configuration

Figure 17. Provision volumes - host group for nonuniform

4. On the Masking View tab, provide the masking view name, as shown in the following figure:

Figure 18. Provision volumes - masking view

5. In the Port Group tab:

For FC, enter the target port WWN. The PowerMax FA port is displayed automatically, as shown in the following figure:

(28)

Configuration

Figure 19. Provision volumes – port group for FC

▪ For iSCSI, enter the target port IQN. The PowerMax target port IQN is displayed, as shown in the following figure:

Figure 20. Provisioning volumes: Host group for iSCSI

6. In the Storage Group tab, as shown in the following figure, select New storage group and enter the storage group name.

(29)

Configuration

Figure 21. Provision volumes - storage group

7. On the Volumes tab, as shown in the following figure, select New volumes and provide the required details about the volumes.

Figure 22. Provision volumes - volumes

8. Click RUN and wait for the workflow to complete successfully, as shown in the following figure:

(30)

Configuration

Figure 23. Provision volumes – complete status

9. Check the PowerMax devices on the VxRail nodes of the VI workload domain cluster.

Only VxRail nodes in AZ1 can see the PowerMax devices, as shown in the following figure:

Figure 24. PowerMax devices shown on VCF AZ1 server

10. Create PowerMax SRDF/Metro:

a. Run the “Create Storage Group SRDF Protection” workflow. b. Enter the source and target array information.

c. In SRDF replication mode, select Active.

The workflow creates the volumes on the target array and establishes the SRDF/Metro pairs, as shown in the following figure:

(31)

Configuration

Figure 25. Create storage group SRDF protection - general

Figure 26. Create storage group SRDF protection - advanced options

11. Configure auto storage provisioning on the remote PowerMax. a. Run the “Provision Volumes to Host” workflow.

(32)

Testing methodology

b. On the Storage Group tab, enter the storage group name that you created in the preceding step in “Existing storage group name.”

If it is a uniform configuration, the PowerMax device paths are doubled, as shown in the following figure:

Figure 27. PowerMax device paths with vMSC uniform configuration

If it is a nonuniform configuration, the AZ2 servers start to see the PowerMax devices, as shown in the following figure:

Figure 28. PowerMax device paths with vMSC nonuniform configuration

Testing methodology

We implemented four test scenarios for our validation. For FC:

• VMware VCF on VxRail with PowerMax SRDF/Metro FC uniform configuration plus NMP

• VMware VCF on VxRail with PowerMax SRDF/Metro FC nonuniform configuration

(33)

Testing methodology For iSCSI:

• VMware VCF on VxRail with PowerMax SRDF/Metro iSCSI uniform configuration plus Dell EMC PowerPath/VE 7.0

• VMware VCF on VxRail with PowerMax SRDF/Metro iSCSI nonuniform configuration plus NMP

We tested the following configurations:

• VM operating system type: Windows 2012/Windows 2016, Red Hat RHEL6/Red Hat RHEL7

• The I/O tool fio run on each VM Functional verification testing

This testing addresses the basic operations that are performed when PowerMax is used as supplementary storage with VMware VCF on VxRail. The following table describes the test cases and the expected results:

Table 6. Functional verification testing

Number Test case Expected result

1 PowerMax SRDF/Metro LUN provision to servers of workload domain

PowerMax SRDF/Metro LUN can be used for

supplementary storage to VI workload domain

2 VMFS5\6 creation on PowerMax SRDF/Metro LUN over 2 TB

Both VMFS5 and VMFS6 datastores are verified on PowerMax SRDF/Metro devices. They all can be read and written to in the VxRail stretched cluster 3 VM deployment, Power On\Off &

Suspend\Resume on PowerMax SRDF/Metro LUN

VM can be deployed on PowerMax SRDF/Metro devices in VI workload domain and all VM operations work properly 4 VM snapshot create and revert and delete on

PowerMax SRDF/Metro LUN

The operation on VM snapshot works well on PowerMax SRDF/Metro device on VI workload domain

5 VM clone on PowerMax SRDF/Metro LUN Clone VMs which are running I/O on PowerMax. SRDF/Metro device works correctly

6 VM (newly created and workload domain-initiated NSX appliance) can perform storage migration bidirectionally between PowerMax SRDF/Metro LUN

VM storage migration works on both the primary storage and supplementary storage of VI the workload domain

Test

configurations

(34)

Testing methodology

7 LUN hot add/remove with PowerMax SRDF/Metro LUN to workload domain cluster node

PowerMax SRDF/Metro devices can be hot added and removed in VI workload domain

Note: The SRDF group is

required to be suspended before you add or remove a device on the existing SRDF group. There is no I/O interruption on PowerMax SRDF/Metro uniform configuration. The I/O on AZ2 is interrupted on PowerMax SRDF/Metro nonuniform configuration.

8 Online device expansion on PowerMax SRDF/Metro LUN in workload domain

Perform online device expansion on any PowerMax site. All sites of

PowerMax/SRDF devices can be updated

simultaneously. 9 VM migration in workload domain cluster on

PowerMax SRDF/Metro

VM migration within the datastore which is supplementary to the VI workload domain,

High availability testing

HA testing helps validate the capability of the solution to avoid a single point of failure from the hardware component port level up to the IDC site level. The following table describes the HA testing that we performed:

Table 7. High availability testing

1 Maintenance work for management domain level HA protection

The VCF on VxRail stretch cluster keeps working without any interruption. 2 Host side HBA port failure triggered I/O path

failover

Host FC/iSCSI connection is partly failed. There is no I/O interruption in such a condition.

3 Storage-side service processor port failure triggered I/O path failover

One PowerMax director is in failure. The I/O does not interrupt because FC zoning and storage provision were also performed on different PowerMax directors to provide a failover path. 4 Storage firmware NDU No I/O interruption during

the storage firmware upgrade.

(35)

Testing methodology

5 VCF Stretch Cluster-WLD AZ1 host unplanned failure triggered HA protection

The powered-on VMs on the AZ1 host can be restarted on the AZ2 host.

The powered-on VMs on AZ2 keep running with no I/O interruption.

6 Network failure between WLD AZ1 hosts and AZ2 hosts

The AZ1 and AZ2 network isolation does not impact the I/O running on a VI workload domain with PowerMax SRDF/Metro. 7 PowerMax SRDF/Metro: unplanned failure on

one of the PowerMax nodes (R1 or R2)

While one PowerMax is dead:

- For vMSC uniform configuration: no I/O interruption while one PowerMax is down. - For vMSC nonuniform configuration: the VMs on the surviving PowerMax continue to run without interruption, the VMs on the dead PowerMax restart on the surviving side with VMware HA PDL enabled after the PowerMax is online, all PowerMax SRDF/Metro devices path are active, and VMs can be migrated between AZ1 and AZ2.

8 PowerMax SRDF/Metro feature test for interlink break triggered VM level HA protection

For uniform vMSC configuration: no I/O interruption on both AZ1 and AZ2.

For nonuniform vMSC configuration: VMs on AZ2 that connect to PowerMax R2 are restarted on AZ1 and connected to PowerMax R1 when HA PDL is enabled on the VI workload domain.

Reliability testing

In general, reliability testing validates whether the components and the whole system are reliable enough with a certain level of stress running on them.

(36)

Testing methodology

Table 8. Reliability testing

1 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to the workload domain

All VM power-on operations can be successfully run simultaneously. 2 Scale testing for long-time stress I/O running in

all VMs

Continually running I/O for more than 24 hours without error

Table 9. Test results for FC

Number Test case FC uniform with NMP

FC nonuniform with PPVE7.0

Pass Pass

3 VM deployment, Power On\Off, and Suspend\Resume on PowerMax SRDF/Metro LUN

Pass Pass

4 VM snapshot create and revert and delete on PowerMax SRDF/Metro LUN

Pass Pass

5 VM clone on PowerMax SRDF/Metro LUN

Pass Pass

6 VM (newly created and workload domain initiated NSX appliance) can do storage migration bidirectionally between PowerMax SRDF/Metro LUN

Pass Pass

7 LUN hot add/remove with PowerMax SRDF/Metro LUN to workload domain cluster node

Pass Pass

9 VM migration in workload domain cluster on PowerMax SRDF/Metro

Pass Pass

10 Host-side HBA port failure triggered I/O path failover

Pass Pass

11 Storage-side SP port failure triggered I/O path failover

Pass. Pass

(37)

Testing methodology

Number Test case FC uniform with NMP

FC nonuniform with PPVE7.0

13 PowerMax SRDF/Metro feature test for interlink break triggered VM level HA protection

Pass Pass

14 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to workload domain

Pass Pass

15 Scale testing for long time stress I/O running in all VMs

Pass Pass

16 VCF stretch cluster-WLD AZ1 host unplanned failure triggered HA protection

Pass Pass

17 Maintenance work for

management domain level HA protection

Pass Pass

19 PowerMax SRDF/Metro: unplanned failure on one of PowerMax Node (R1 or R2)

Pass Pass

Table 10. Test results for iSCSI

Number Test case iSCSI uniform with PPVE7.0

iSCSI nonuniform with NMP

Pass Pass

3 VM deployment, Power On/Off and Suspend/Resume on PowerMax SRDF/Metro LUN

Pass Pass

4 VM snapshot create and revert and delete on PowerMax SRDF/Metro LUN

Pass Pass

5 VM clone on PowerMax SRDF/Metro LUN Pass Pass 6 VM (newly created and workload domain

initiated NSX appliance) can perform storage migration bidirectionally between PowerMax SRDF/Metro LUN

Pass Pass

7 LUN hot add/remove with PowerMax

SRDF/Metro LUN to workload domain cluster node

Pass Pass

(38)

Conclusion

Number Test case iSCSI uniform with PPVE7.0

iSCSI nonuniform with NMP

9 VM migration in workload domain cluster on PowerMax SRDF/Metro

Pass Pass

10 Host-side HBA port failure triggered I/O path failover

Pass Pass

11 Storage-side SP port failure triggered /IO path failover

Pass Pass

12 Storage firmware NDU Pass Pass 13 PowerMax SRDF/Metro feature test for

interlink break triggered VM level HA protection

Pass Pass

14 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to workload domain

Pass Pass

15 Scale testing for long-time stress I/O running in all VMs

Pass Pass

16 VCF stretch cluster-WLD AZ1 host unplanned failure triggered HA protection

Pass Pass

17 Maintenance work for management domain level HA protection

Pass Pass

19 PowerMax SRDF/Metro: unplanned failure on one of PowerMax Node (R1 or R2)

Pass Pass

Conclusion

This white paper describes how Dell EMC engineers integrated VMware Cloud Foundation on VxRail with PowerMax SRDF/Metro and the design configuration steps that they took to automatically provision PowerMax storage by using the PowerMax vRO plug-in. The paper validates that the Cloud Foundation on VxRail solution functions as expected in both a PowerMax uniform vMSC configuration and a nonuniform vMSC configuration by passing all the designed test cases. This reference architecture validation demonstrates the power of the Dell Technologies portfolio to provide customers with modern cloud infrastructure technologies that deliver the highest levels of application availability for business-critical and mission-critical applications running in their private clouds.

(39)

References

The following documents on the Dell Technologies online support website provide additional information. Access to these documents depends on your login credentials. If you do not have access to a document, contact your Dell Technologies representative:

• Dell EMC PowerMax and VMAX All Flash: SRDF/Metro Overview and Best Practices

• Dell EMC Unisphere for PowerMax Installation Guide • vRO Plug-in for Dell EMC PowerMax Installation Guide

• BEST PRACTICES FOR USING Dell EMC SRDF/METRO IN A VMWARE VSPHERE METRO STORAGE CLUSER

• VMware vSphere Metro Storage Cluster (vMSC) with Dell EMC PowerMax and VMAX SRDF/Metro (Partner Verified and Supported) (2134684)

• VMware Cloud Foundation 3.9.1 on Dell EMC VxRail Release Notes • VxRail-Appliance-Software-4.7.x-Release-Notes

• Stretch a Cluster for NSX-V in VMware Cloud Foundation 3.9.1 • Deploy vRealize Automation in Cloud Foundation