Cloud Data Management Interface (CDMI TM ) Use Cases: Leveraging CDMI Features. May 2013

(1)

Cloud Data Management Interface (CDMI

TM

₎

Use Cases: Leveraging CDMI Features

May 2013

Authored by Seth

Mason

Cisco Systems Inc.

(2)

Overview

Whitepapers published by the Storage Networking Industry Association (SNIA) Cloud Storage Initiative (CSI) are intended to provide information to storage professionals on aspects of standards for storage.

This whitepaper is intended for developers and architects of cloud storage services who are investigating applicability and implementation of cloud storage alternatives. It provides examples or use cases that leverage cloud-based storage and the SNIA Cloud Data Management

Interface (CDMITM) interface. This whitepaper concentrates not only on generic use cases of

cloud-based storage, but on use specific to CDMIand the use of CDMIfeatures which at time of

publication are not supported by other interfaces such as Amazon’s S31 cloud storage interface.

This whitepaper may also serve as a guide to implementers of systems that support CDMI. SNIA’s CDMI specification outlines the methods and objects required to access cloud-based storage residing either within a private datacenter or a remote location maintained by a service provider. At the time of writing, the current version of CDMI is 1.0.2 and is available at

http://snia.org/CDMI.

The use cases in this whitepaper will focus on two main areas which apply to either consumers of cloud storage or the implementers of a cloud storage application or service: queues and complex queries.

Readers should note that the SNIA CDMI specification has been published by the International Standards Organization and the International Electrotechnical Commission (ISO/IEC) as an ISO/IEC International Standard. Readers may purchase the standard through the ISO store

(http://www.iso.org/iso/store.htm). The ISO/IEC CDMI standard number is

ISO/IEC 17826-2012.

(3)

Deploying Queues for Multi-Cloud Disaster Recovery

While cloud-based storage is achieving rapid acceptance and market growth, many storage administrators and architects continue to deploy storage internally to their organization to keep it local to the applications hosted within their data centers. Reasons for this include security and improving the performance of the data access. Some, however, are gaining the benefits of lower-cost cloud storage from service providers by opting to build private-public hybrid clouds for disaster recovery. The use of a hybrid cloud enables the cloud administrator to replicate the data to an offsite storage cloud maintained by a storage cloud service provider. This example of

leveraging CDMIwith respect to a hybrid cloud will be used to illustrate CDMIimplementation of

queues.

By connecting the local storage cloud to the service provider’s storage cloud, all primary access is still performed within the company’s datacenter. However, long term backup and/or archiving is provided by the service provider’s storage cloud. This reduces infrastructure costs. Further, the

use of the CDMIinterface standard enables users to leverage a consistent access methodology

throughout the information lifecycle.

While conventional technologies provide data duplication within the local storage cloud or replication to multiple sites, the complexity of replicating to an service provider’s cloud is the responsibility of the cloud administrator or architect. The service provider’s cloud is maintained as a separate entity. Local cloud administrators are required to set up policies for different types of data, objects and/or containers. For example, some low priority data could be kept locally, medium priority data could be replicated once a week, high priority data replicated daily, and business critical data could be set up to be replicated hourly. These customer requirements could be negotiated as part of a chargeback system implemented by the local storage cloud

administrators.

To illustrate how the storage cloud would use CDMIqueues, consider a local cloud administrator

who wants to create a backend or replication infrastructure within a cloud dedicated to moving objects from the local storage cloud to the service provider’s storage cloud. He can create

multiple FIFO queues (ex: medium, high, business critical), each automatically populated with the objects that have been added from the local cloud. It would also be advisable to create another set of three FIFO queues corresponding to the aforementioned three queues to facilitate objects that are to be deleted, or expired from the service provider’s cloud after a specified period of time. Using the queues they can be temporarily stored in preparation for expiration and/or deletion. At a predetermined time (hourly, daily or weekly), the local cloud administrator would copy, or de-queue, the objects from the backup queue and copy them to the service provider’s cloud, keeping the objects attributes and meta-data intact. By keeping the meta-data intact, such as the object’s classification, lifecycle policy information, etc. during a disaster recovery operation, the storage cloud administrator could efficiently retrieve the objects in order of classification.

While the CDMI specification defines the use of queues, it leaves it up to the developer or cloud

vendor how the specific queues are to be implemented in a cloud storage infrastructure. It is this flexibility that enables cloud storage vendors to differentiate themselves, while still conforming to

the CDMIspecification. For example, one vendor could implement queues in such a way where

(4)

taking up very little additional disk capacity, beyond the pointers themselves. An alternative implementation would have queues on a dedicated infrastructure such that the objects in the queues are copied to a dedicated set of storage nodes or disks. The example model discussed in this section could illustrate these implementation methods by including business critical queue objects in a dedicated infrastructure while the high and medium priority objects are relegated to a pointer based infrastructure.

While this example of using multiple queues seems simple, it is possible to extend the

requirements of the queues based upon any number of or combination of information and/or data requirements. Such requirements could be distance, as in “this object must be replicated to a facility over 100km away” or “must remain within state, country or regional boundary” or “data must be replicated or retrievable from locations on different power grids” or even “must be replicated to two different cloud storage service providers”. This would require the local cloud administrator to create more queues that are filled with objects from the results of a more complex nature and more importantly, resultant policies based upon business and perhaps regulatory input from sources beyond the scope of the local cloud administrator and the datacenter.

The uses for CDMIqueues are quite powerful in nature as it enables a storage cloud

administrator to easily access a set of data in the simplest method possible, reading from a queue.

Locating Objects through Complex Queries

As the need for unstructured data grows within an organization, quite often from the millions of objects to the billions and trillions of objects, the need to find better ways to retrieve those objects quickly also grows. An application owner that is storing data or objects in the storage cloud needs to be able to perform a complex query against the storage cloud and have the cloud return objects

that meet the criteria. The CDMIspecification supports the ability to query the storage cloud

based upon a regular expression.

The advantage of being able to query the cloud itself is that as the size of the storage cloud infrastructure scales, so does its ability to execute that query. This would not be the case with storing the data in an external database, or having to scan the storage cloud itself and filter out the unmatched objects.

In some cities, power utilities have begun to deploy smart power meters. These power meters report the usage of electricity in the home or business back through a wireless network. This could be further extended to have a set of devices within the home or business report to the meter what their specific usage is. So for example, an electric furnace or air conditioner could report exactly when it goes on, for how long, how much power is being drawn. This could then be reported back to the power utility company so that eventually the consumer could see how much power they use within a given period of time and what their furnace or air-conditioner’s usage is. While this example so far is quite simple, it’s in the subsequent use of the data for mining of the

data by the power utility company whereregular expressions prove fruitful.

(5)

heating/cooling etc.) and more. This data would be added to the power consumption object, providing finer power consumption granularity, down to the appliance itself.

CDMIsupport for containers within containers makes it possible for the power utility company to

implement storage cloud applications with hierarchies of containers (i.e. state/city/zipcode/street address) designed to aid in organizing data more aligned to company objectives.

Meters can potentially report daily or even hourly power readings, leading to the power utility company filling their storage cloud with millions of objects per day. While the objects themselves are not large, the number of objects could be unmanageable if implemented in other means. Utilizing this data, the power company could decide to provide incentives or detailed reporting to a specific subset of consumers.

An application owner at the utility company could build a query to return only those addresses that meet the following criteria to find home owners that are wasting electricity by leaving their furnace or air-conditioner on during the work day (Monday through Friday from 9am until 5pm):

1. Home residences and not commercial properties 2. In zip-code #####.

3. For days Monday, Tuesday, Wednesday, Thursday, Friday 4. For the hours between 9 am and 5 pm.

5. Use at least X Watts of power.

6. Weather temperature is > 65degrees F. 7. Weather temperature is < 80degrees F.

Based on this information, the power utility company could then provide notification to the

consumer that could suggest ways for the consumer to use power more efficiently to reduce their bill. They could also do follow up queries, such as querying the storage cloud to determine the age of the furnace or air-conditioner, perhaps listing only those residences with an appliance that is over a certain age, for the purpose of providing further incentives for consumers to replace their aging appliances.

This model could be deployed internally to the power utility company, but since power utilities are in the business of generating power and not in the business of IT, they might migrate to a CDMI-based storage cloud provider, so they can concentrate on their core business of generating and reporting on power usage. Furthermore, third party application developers targeting the power utility companies could develop and provide applications that utility companies can use to support more end-to-end solutions. The power company would then be free to choose from a wide array of CDMI-based storage cloud providers based upon the utility’s requirements and budget.

Let's take it one step further. If the power utility was to build upon CDMIconcepts of queues,

(6)

SNIA’s Cloud Technical Working Group was motivated to design CDMIwith flexible features such

as the ability to do complex queries while enabling an open interface for accessing cloud-based storage, thus providing application owners the ability to continually scale their applications to meet demands. This scaling will not be just be from the traditional access speeds and capacity as seen throughout the history of storage, but in scaling the intelligence of the storage device itself such that it is aware of the data contained within it. It is this intelligent scaling that will provide the next generation of applications the ability to store and find the objects within the ocean of data

(7)

About the Author

Seth Mason is a Distinguished Services Engineer and a cornerstone in the areas of Storage and Unified Networking, a thought leader in Cisco’s advancement in the Data Center, and a founding member of Cisco’s storage services practice.

Seth has more than two decades of experience focused on the datacenter. He is the co-author for The Cisco Storage Networking Cookbook, and the Cisco MDS Interoperability and Configuration Guide. He is the co-inventor for the Cisco MDS feature, Device Aliases as well as tools used to analyze and optimize customer’s networks. His current responsibilities center on expanding customer networks to include FCoE and network based analytics. He is long time speaker at Cisco Live on topics such as Unified Fabrics, table-talks and Meet the Engineer. He is one of the original authors of the CCIE:SAN exam and continues to contribute to its’ success.

Seth holds a Bachelors of Computer Engineering from Auburn University.

About the CSI

The SNIA Cloud Storage Initiative (CSI) was created to foster the growth and success of the market for cloud storage. Members of the SNIA CSI work together to educate the vendor and user communities about cloud storage, perform market outreach that highlights the virtues of cloud storage, collaborate with other industry associations on cloud storage technical work, and coordinate with SNIA Regional Affiliates to ensure that the results of CSI activities are felt worldwide. The CSI, along with 140 individuals from more than 30 organizations, promotes the

adoption of standardization through the Cloud Data Management Interface (CDMITM) standard

specification. For more information or to get involved, visit the SNIA CSI website at www.snia.org/cloud.

About the SNIA

The Storage Networking Industry Association (SNIA) is a not-for-profit global organization made up of some 400 member companies spanning virtually the entire storage industry. SNIA’s mission is to lead the storage industry worldwide in developing and promoting standards, technologies, and educational services to empower organizations in the management of

information. To this end, the SNIA is uniquely committed to delivering standards, education, and services that will propel open storage networking solutions into the broader market. For

(8)

Use of this document

The SNIA hereby grants permission for individuals to use this document for personal use only, and for corporations and other business entities to use this document for internal use only (including internal copying, distribution, and display) provided that:

• Any text, diagram, chart, table, or definition reproduced shall be reproduced in its entirety with no alteration

• Any document, printed or electronic, in which material from this document (or any portion hereof) is reproduced shall acknowledge the SNIA copyright on that material, and shall credit the SNIA for granting permission for its reuse

Other than as explicitly provided above, you may not make any commercial use of this

document, sell any excerpt or this entire document, or distribute this document to third parties. All rights not explicitly granted are expressly reserved to SNIA.