White Paper
EMC Solutions Group
Abstract
This solution illustrates how public and private cloud providers can use EMC
®Symmetrix
®VMAX
®Cloud Edition as a foundation for their as-a-service offerings by integrating it into their existing multitenant cloud infrastructure based on VMware vCloud.
September 2013
INTEGRATING CLOUD ORCHESTRATION WITH EMC SYMMETRIX VMAX CLOUD EDITION
REST APIs
Provisioning storage using EMC Symmetrix VMAX Cloud Edition
Using REST APIs for integration with VMware vCloud
Deploying applications in a multitenant environment
Copyright © 2013 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of
merchantability or fitness for a particular purpose.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 3
Table of contents
Executive summary ... 4
Business case ... 4
Solution overview ... 4
Key results/ recommendations ... 4
Introduction... 6
Purpose ... 6
Scope ... 6
Audience ... 6
Terminology ... 6
Technology overview ... 8
VMAX Cloud Edition ... 8
VMAX Cloud Edition service levels ... 11
VMware vCenter Orchestrator ... 13
VMware vCenter Chargeback ... 14
VMware WaveMaker ... 15
VMware vCloud Director ... 16
VMware vSphere ... 16
Solution use case ... 18
Configuration ... 18
Service provider role ... 19
Tenant administrator role ... 24
Tenant user role ... 28
Optional configuration ... 30
VMware vCloud Integration Manager ... 30
Conclusion ... 33
Summary ... 33
Findings ... 33
References... 34
White papers ... 34
Other documentation ... 34
Executive summary
Service providers, enterprises, and mid-market businesses are increasingly driving a new “as-a-service” delivery model for IT, becoming providers of public and private clouds. As their customer base expands, they are faced with the challenges of reducing complexity, driving automation, delivering secure multitenancy, improving TCO, and simplifying IT planning. As the concept of cloud computing becomes a reality, application owners are quickly changing their thinking from infrastructure- based deployments to service-level-based deployments. Less consideration is given to the mechanics of the underlying infrastructure and more to performance levels that can be delivered predictably to their business-critical applications regardless of the underlying components.
Cloud providers—that is, service providers serving multiple tenants in a shared public cloud, or enterprise IT organizations serving multiple internal business units—need to be able to deliver the service levels their customers are requesting in the most
automated and efficient manner possible. Offering cloud providers an as-a-service platform that delivers automated service-level provisioning with automatic metering, reporting, and billing is an important step toward realizing IT-as-a-service (ITaaS) efficiency. Offering the platform in a way that can be integrated easily into an existing cloud provider environment is critical to realizing this efficiency.
This solution, developed by EMC Solutions Group, shows how cloud providers can use a shared EMC
®Symmetrix
®VMAX
®Cloud Edition storage array to host their own private clouds as well as provide public services to external customers. Cloud
providers can integrate VMAX Cloud Edition into their new and existing “as-a-service”
environment using the REST APIs provided. The VMAX Cloud Edition REST APIs allow the cloud provider to build automation workflows leveraging the orchestration and web portal tools that they have in place to execute common storage management and provisioning tasks.
In the use case demonstrated in this paper, a cloud administrator provider gets various service levels of storage from the cloud provider and uses them in a VMware vCloud environment. The cloud administrator then sells the virtual data centers as products using vCloud Integration Manager and a custom portal developed using VMware WaveMaker. Automation is performed using the VMware vCenter
Orchestrator engine. VMAX Cloud Edition billing reports are used to determine the base rates for the storage, and VMware vCenter Chargeback is used to create the detailed billing report based on the actual use of various components.
This white paper demonstrates how a cloud provider can use the REST-based APIs available in VMAX Cloud Edition to integrate the platform into its existing VMware Business case
Solution overview
Key results/
recommendations
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 5 creating a Provider virtual data center (vDC) that includes all storage profiles for tenant use.
The cloud provider should also consider using VMAX Cloud Edition data collection to
facilitate reporting and billing by VMware vCenter Chargeback.
Introduction
The purpose of this white paper is to provide cloud providers with a detailed example of how they can use VMAX Cloud Edition in the most automated and efficient manner within the parameters of their existing infrastructure based on VMware vCloud. After reading this white paper, cloud providers should understand the benefits of
integrating VMAX Cloud Edition into their environments using the available REST APIs to automate storage provisioning and implement multilevel storage for deployments of business-critical applications.
The scope of this paper is to provide an overview of an existing multitenant environment based on VMware vCloud and a detailed description of how a VMAX Cloud Edition array can be programmatically integrated into that environment to provide automated storage provisioning for the cloud. This paper also demonstrates how management can be delegated among a cloud provider, customer, and tenant user.
The cloud storage in this paper encompasses the Fibre Channel SAN-based block storage provided by VMAX Cloud Edition, the VMFS datastores that are based on those volumes, and the vApps that consume the storage as virtual disks within the organization’s vDCs.
This white paper is intended for technical engineering staff responsible for deploying and configuring as-a-service offerings in a private, public, and/or hybrid cloud environment. It is assumed that the reader has a strong background in virtualization technologies and overall infrastructure architecture knowledge. With Symmetrix VMAX Cloud Edition management and automation, only a cursory knowledge of storage technologies is necessary, as Symmetrix VMAX Cloud Edition is built specifically to abstract the underlying mechanics of typical storage technologies.
Finally, since a large part of this solution involves API integration, a strong knowledge of REST-based APIs and development tools and frameworks is recommended.
This paper includes the terminology defined in Table 1.
Table 1. Terminology
Term Definition
Application programming interface (API)
A source-code-based specification intended to be used by software components as an interface to communicate with each other.
Organization The unit of multitenancy that represents a single logical Purpose
Scope
Audience
Terminology
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 7
Term Definition
Provider virtual data center (Provider vDC)
A grouping of compute and storage resources from a single vCenter server instance. A Provider vDC consists of a pool of physical compute resources and one or more datastores.
Multiple organizations can share Provider vDC resources.
REST API Representational State Transfer application programming interface. REST API defines a set of functions that enable developers to send requests to, and receive responses from, an associated API service through HTTP.
SAN fabric zoning The logical grouping of physical interfaces in a storage area network that enables communication between a source (server) and a target (storage array).
Storage tiering The assignment of data to different types (tiers) of storage media to meet the service levels and thus reduce total storage cost.
vApp A container for a distributed software solution. vApp, which is the standard unit of deployment in vCloud Director, consists of one or more virtual machines and can be imported or exported as an Open Virtualization Format (OVF) package.
VMware vCloud Integration Manager (vCIM)
Software that enables virtual data centers to be sold as
products and enables service provider administrators to
maintain the distributors and resellers. vCIM is available to
service providers who are part of the VMware Service
Provider Program (VSPP).
Technology overview
VMAX Cloud Edition enables cloud providers to automate storage provisioning to host private and public clouds as well as legacy applications in a physical environment.
Each tenant can have its own view of the management portal and billing information.
Tenants are billed based on service levels of storage, and those levels are based on performance characteristics. VMAX Cloud Edition also provides flexibility in
automating the provisioning of storage to hosts, including defining and administering SAN fabric zones. VMAX Cloud Edition provides:
Secure multitenancy
Role-based access control (RBAC) determines the level of access for the provider and tenants.
A web portal enables tenant self-service, which reduces the need for provider intervention by enabling delegation of day-to-day management tasks, such as storage provisioning and de-provisioning, to the tenants.
The portal manages multiple instances of VMAX Cloud Edition and can be branded per provider, as shown in Figure 1.
Figure 1. Tenant portal view VMAX Cloud
Edition
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 9
Versatile web portal
Users manage their storage assets by submitting service requests to the EMC data center through the web portal, as shown in Figure 2. Interfaces are simplified and abstracted for tenant use. VMAX Cloud Edition automatically takes these logical requests and intelligently translates them into specific resource allocation requests, based on EMC-proven best practices. The requests are then executed against specific storage assets.
Figure 2. Tenant Admin portal view
The web portal also provides reporting, trending, and monitoring capabilities.
Usage reports are tenant-based to facilitate chargeback.
VMAX Cloud Edition storage service catalog
The levels of storage within VMAX Cloud Edition are described as logical service levels as a part of a service catalog. Each service level is described logically based on its availability and performance characteristics. Users can provision from the service level that best matches their specific application needs, as shown in Figure 3.
Figure 3. VMAX Cloud Edition storage service catalog
Additional availability profiles can be added to each service level to facilitate business continuity, disaster recovery, mobility, and high availability.
REST APIs
In addition to providing a web-portal user interface, VMAX Cloud Edition supports REST interfaces, as shown in Figure 4. This enables cloud providers to programmatically integrate VMAX Cloud Edition capabilities directly into their existing orchestration infrastructure. For ease of integration into existing cloud provider environments, VMAX Cloud Edition also provides Web Application Description Language (WADL), which can be accessed at
http://{IPAddress}/restapi/application.wadl. It supports both JSON and XML
outputs.
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 11 Figure 4. VMAX Cloud Edition REST APIs
VMAX Cloud Edition provides storage service levels of varying availability, capacity, and performance. Each service level is a pre-engineered mix of different drive types, as shown in Figure 5, individually weighted respective to their own quality of service.
With different ratios of drive types in each service level, some service levels are weighted in favor of capacity and others in favor of overall performance, which enables cloud providers to charge appropriate prices per level while still maintaining performance and operational benefits.
VMAX Cloud
Edition service
levels
Figure 5. Examples of storage service levels
The service levels provide the highest I/O density at the highest-available level where workloads demand it. The I/O density decreases as the service levels move toward serving less performance-intensive workloads where cost is of more importance.
There are multiple bands available in each of these service levels, allowing for flexibility in pricing and performance.
The service levels illustrated in Figure 3 on page 10 represent a subset of the storage service levels available on VMAX Cloud Edition. Variants of each of these service levels are available from a range of drive packs based on fixed policy mixes.
The tenants are provided with the information about various service levels, expected
response times, IOPS, and typical use cases. Even if they fail to choose the correct
level during provisioning, they can change the service level of the volume at a later
time without impact to their applications. Figure 6 shows how a service level can be
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 13 changed after an application has been deployed on the storage.
Figure 6. Portal service-level change utility
Appropriate cost models and rate factors can be applied to these service levels. To facilitate chargeback, allocation reports can be generated on details such as storage consumption.
VMAX Cloud Edition performs the SAN fabric zoning, configuring the number of storage paths appropriate for the chosen service level.
VMware vCenter Orchestrator is an IT process automation engine that helps automate cloud processes. It simplifies the integration of VMware vCloud Suite with third-party products and management systems such as VMAX Cloud Edition. vCenter
Orchestrator enables workflow definition to automate common tasks with predefined libraries. It also can be extended using third-party plug-ins.
In our test environment, we used the REST plug-in for vCenter Orchestrator to define the REST hosts of VMAX Cloud Edition, vCIM, and vCenter Chargeback, and we used those web services calls in vCenter Orchestrator workflows. We also used workflows provided by VMware plug-ins for vCenter, vCloud Director, and vCenter Chargeback and extended those workflows as needed. Figure7 depicts a standard vCenter Orchestrator workflow used in our use-case environment.
VMware vCenter
Orchestrator
Figure 7. vCenter Orchestrator sequence workflow
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 15 Chargeback costing options that can be used to provide usage and billing
information.
Figure 8. vCenter Chargeback
VMware WaveMaker provides a rapid Web 2.0 application development environment.
It includes a WYSIWYG development studio for creating web applications. Using WaveMaker, we created a sample cloud provider portal to demonstrate the integration capabilities of VMAX Cloud Edition. WaveMaker interacts with vCenter Orchestrator web services to execute workflows and retrieves its output in XML format. Figure 9 shows the WaveMaker configuration options available for the portal page.
VMware
WaveMaker
Figure 9. WaveMaker portal configuration page
VMware vCloud Director provides a portal for cloud providers to use to define virtual data centers. A tenant administrator can use the portal to define service catalogs, users, and policies for those virtual data centers. Tenant users can use the portal to create or consume vApps in those virtual data centers.
vCloud Director 5.1 allows the use of multiple storage profiles within the same data center, and virtual machines in the vApp can be deployed to specific storage profiles.
VMware vSphere
®provides the virtualization layer to abstract the underlying hardware resources. VMAX Cloud Edition supports vSphere Storage APIs for Array Integration (VAAI), which provides hardware acceleration for vSphere, by offloading certain storage operations, like cloning and zeroing, to the array for greater efficiency.
Figure 10 shows that the datastores provisioned to the vCenter support hardware acceleration.
VMware vCloud Director
VMware vSphere
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 17 Figure 10. View of vSphere datastores with VAAI support
VAAI support can also be confirmed using esxtop.
Solution use case
VMAX Cloud Edition can be used by any cloud provider to allow tenants to host private, public, and/or hybrid clouds. In our solution use case, a vCloud provider who uses VMAX Cloud Edition creates virtual data centers and sells them as a product.
The cloud provider environment described in this white paper and depicted in Figure 11 is based on VMware vCloud and includes vCloud Director, vCenter Orchestrator, vCenter Chargeback, and vSphere. Additionally, vCIM is used to automate sign-up and provisioning of vCloud Director tenant environments. Finally, administrator and user access is provided by a front-end portal based on WaveMaker from VMware.
Figure 11. Logical depiction of cloud provider environment
The cloud provider using VMAX Cloud Edition gets shared storage of various levels.
Using this shared storage, the cloud provider creates Provider vDCs and publishes product templates using vCloud Director. Customers then order virtual data centers as products and deploy their applications within those virtual data centers.
VMAX Cloud Edition provides allocation information for various service-level volumes.
The cloud provider uses that information to determine the base rate for those
volumes. This solution uses vCenter Chargeback to generate billing reports based on
Configuration
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 19 Using administrator credentials, the cloud provider logs in to the portal, as shown in Figure 12. The portal in our case was created using VMware WaveMaker.
Figure 12. Service provider portal login window
Once logged in, the cloud provider can see the list of tenants and can add and remove tenants. Figure 13 shows the cloud provider view of the sample portal.
Figure 13. Cloud provider view of portal Service provider
role
Adding a tenant initiates execution of a vCenter Orchestrator workflow to add a customer organization in vCloud Director. The workflow also creates an administrator account within that organization.
Figure 14 shows the organizations created in the test environment vCloud Director instance.
Figure 14. vCloud Director organizations
As shown in Figure 15, the cloud provider can list the product templates that equate to service levels of storage provisioned from VMAX Cloud Edition.
Figure 15. Product templates provided by cloud provider through VMAX Cloud Edition
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 21 In addition, the cloud provider can add new product templates by specifying the required storage size and selecting the service level, as shown in Figure 16.
Figure 16. Create Product Template Dialog
This action initiates execution of the following vCenter workflow:
1. Create a volume on the appropriate service level on VMAX Cloud Edition.
2. Create the fabric zone if needed.
3. Present the newly created volume to the vSphere host.
4. Rescan the vSphere host.
5. Create the datastore.
The datastore is then used to create the Provider vDC in vCloud Director.
Figure 17 shows the Volume Create request on VMAX Cloud Edition.
Figure 17. VMAX Cloud Edition Volume Create request
Integrating Cloud Orchestration with EMC Symmetrix VMAX Cloud Edition REST APIs 23 Note: Previous versions of vCloud Director require a separate vDC for every service
level. With vCloud Director 1.5, a single Provider vDC can have multiple storage profiles within it if necessary. Having multiple storage profiles within the same vDC can simplify configurations in the event a tenant ever needs to change the service level of a volume.
The cloud provider can view a tenant’s consumption information and generate a billing report, as shown in Figure 18.
Figure 18. Billing Report request
The reports generated can be viewed using the WaveMaker portal as well as under Archived Reports in vCenter Chargeback, where they can be sent by email, generated in PDF, or exported as CSV files or MS Word documents. Figure 19 shows the Archived Reports list in vCenter Chargeback.
Figure 19. vCenter Chargeback Archived Reports
The tenant administrator can log in to the cloud provider portal and view the storage service levels (product templates) available to provision.
Figure 20 shows the username format that a tenant administrator uses to log in.
