CTERA Cloud Storage Platform
Architecture
Whitepaper by CTERA Networks
Highlights
How unstructured data growth drives cloud storage adoption
The ‘killer apps’ of cloud storage reviewed
Putting cloud storage to work: building blocks and approaches
The need for a unified, interoperable, multi-application platform
Growth of Unstructured Data
The rapid growth of storage consumption, particularly for unstructured data, has driven organizations to adopt cost-effective cloud storage solutions as a substitute or complement to more expensive block or file storage systems. Beyond cost savings, cloud storage offers many benefits to organizations, such as: elastic use of storage to match changing demands, usage metering and pay-per-use cost models, built-in geographical redundancy, global name space, flexible data access methods, and the ability to easily shift data between private and public storage spaces.
Small businesses, enterprises, and government organizations take advantage of cloud storage applications such as backup, file sharing and mobile access. Service providers have also recognized the opportunity and many offer cloud-based storage services to their customers.
The first step in creating a cloud storage offering is having the right infrastructure (deployed in-house, or hosted), and for many applications that infrastructure is object storage. However, the storage infrastructure, on its own, is not sufficient to create and deliver the storage-based applications that users need.
As with any emerging technology space, a plethora of vendors have developed tools to enable specific cloud storage applications. Yet IT organizations and service providers are struggling with the integration of disparate tools into a consistent set of cloud storage services, in an environment fraught with immature technologies and unclear standards. A different approach is needed: a cloud storage platform that supports multiple storage applications in a consistent, integrated and scalable fashion.
‘Killer Apps’ of Cloud Storage
Cloud storage has proven to be a cost effective and
scalable solution for several types of storage
applications:
Backup: Cloud storage is ideally suited for backing up user-generated files, and is an ideal replacement for tape backup and off-site tape storage services.
File Storage: Traditionally, enterprise branch offices have local file servers in each office, for storing local, unstructured data. As their business scales, many enterprises now recognize that this approach creates a maintenance and support nightmare. Cloud storage gateways are an ideal substitute for on-site file servers, providing an easy-to-manage, disaster-proof alternative, while retaining compatibility with existing applications – without sacrificing performance.
File Sharing: Cloud storage can be used to create a simple, intuitive file sharing environment, whereby employees can grant secure file access to team members and guest users.
Mobile Access: Many organizations have adopted ‘bring your own device’ (BYOD) policies, tolerating or encouraging employee use of personal smartphones and tablets to access their corporate data. Cloud storage can be used to support BYOD data access, enabling users to access their files anytime, anywhere.
Archival: A growing number of government regulations (e.g., for financial services, healthcare) require companies and agencies to retain records for extended periods of time. Cloud storage offers a cost-effective and disaster proof way to archive such data.
Cloud Storage Building Blocks
Deploying a cloud storage solution involves integrating key elements, and addressing several key issues: Storage infrastructure: Since cloud storage is often
used for storing ‘unstructured data’ (e.g., files), it is often implemented using object storage architecture. Object storage excels in storing and retrieving variable size objects along with their associated attributes. It typically has application programming interfaces (APIs) that support creation, retrieval and manipulation of stored objects. The industry standard for object storage APIs has evolved around the HTTP protocol, and is commonly referred to as REST API. Storage clients: The devices that access objects (or
files) stored in the cloud. Those typically include users’ workstations, departmental servers and mobile devices.
Connectivity: Most storage clients support traditional
network protocols for accessing files remotely, for example: CIFS (Microsoft Windows), NFS (Linux/Unix) and AFS (Apple Macs). A bridge is needed between these traditional file access protocols and the new cloud storage APIs.
Security: Since cloud storage usually resides off premises, data exchanged between storage clients and the cloud must be protected (e.g. encrypted) both in transit and at rest.
Performance: For end-users to adopt cloud services, they must be able to access their files with reasonable throughput and low latency. Therefore, any solution that moves primary data to the cloud must employ clever caching and tiering mechanisms. Additionally, the volume of data transferred to/from the cloud should be minimized to avoid taxing of network resources and bandwidth, for example by employing WAN optimization, data de-duplication and compression.
Reliability: For end-users to adopt cloud storage services, they must be able to rely on availability of data, even during inevitable Internet connectivity or service disruptions. To reduce the impact of such failures, it is crucial to use a hybrid approach such as on-site storage or pinning of the most important data to be always available locally.
Management: A single cloud storage pool may be used by thousands of sites and numerous users. Any cloud storage service would require sophisticated and scalable management utilities that support quick service provisioning, real-time monitoring and simplified maintenance.
Putting the Pieces Together
Service providers and enterprises who are considering building a cloud storage service (external or internal) can adopt any of the following approaches:
Do It Yourself (DIY): Picking individual cloud storage
building blocks and putting them together. This involves deploying specific storage infrastructure, installing protocol conversion gateways, creating a management framework, developing individual storage services, and integrating the entire system. The advantage of this approach is having full control over the architecture and functionality of the cloud storage service. However, this approach requires a high degree of know-how, significant investment in development and massive integration efforts, with operational costs that are very high (software maintenance, upgrades etc.), making it an unrealistic option for most.
Best-of-breed solutions: Implementing vendor supplied systems for each particular cloud storage application. For example, deploying one system that supports backup, another that offers file sharing services and yet a third to handle mobile access. The main advantage of this approach is the ability to select an optimal solution per each storage application. The challenge involves ‘integration’, or lack thereof, between disparate systems. Both users and administrators are required to deal with multiple interfaces, depending on the application they need to access, and a larger footprint on end-point devices. Unified platform: Deploying a services platform that
What’s in a Platform?
The adoption of new technology typically involves progression from do-it-yourself systems, through best-of-breed tools till finally reaching a platform approach. The cloud storage market has evolved quite rapidly, reaching a maturity level where the platform approach is not only feasible, but delivers benefits that are impossible to achieve using other approaches:
Back-end storage interoperability: Customers may
choose to implement their own private cloud storage infrastructure, or tap into a public cloud storage service. A cloud storage platform should support any combination of private and public backend storage, and interoperate with numerous storage vendors and services, ensuring flexibility for the customer with no vendor/provider lock-in.
Flexible deployment scenarios: Cloud storage
applications may be deployed in small businesses, government agencies or large enterprises with multiple remote offices/branch offices (ROBOs). The data may be accessed from desktops, laptops, web browsers, smartphones or tablets. Users may reside in a corporate office, home, or be on the road. They may include permanent employees as well as consultants, partners or customers. A cloud storage platform must therefore successfully address and support all these deployment scenarios.
IT infrastructure integration: Cloud storage is not
deployed in a vacuum. Service providers and enterprises have significant investments in existing IT infrastructure. Management systems, authentication services, billing systems, and self-service app stores are examples of infrastructure already deployed in datacenters. A cloud storage platform should be able to interface and interoperate with existing IT infrastructure, both at the data center and remote offices.
Scalability: A cloud storage service may be used by a large number of businesses, thousands of corporate remote offices, and numerous end-users. A cloud storage platform must therefore scale from a few to 100,000’s of users and devices. Since demand for storage constantly grows, the platform should scale to virtually any size of back-end storage.
Security: One of the key concerns associated with cloud storage adoption is data security. The cloud storage platform must protect, through encryption, data stored on the cloud storage infrastructure (i.e. at-rest), as well as protect data transferred to/from storage clients and storage infrastructure (i.e. in transit). User authentication procedures and data access rights must be strictly controlled. The platform should also ensure data integrity, to prevent accidental data corruption or unauthorized tampering. Lastly, all data, metadata, encryption keys and access privilege data should all be kept behind the firewall.
Performance: Cloud storage applications involve large volume data transfer (e.g. during backup) and remote data access (e.g. from mobile devices or remote sites). Successful adoption of cloud storage requires that access performance, network load, and storage size requirements are optimized. To that end, the cloud storage platform must employ techniques such as de-duplication, compression and bandwidth throttling.
Manageability: IT teams must be able to easily provision, monitor and maintain the cloud storage service. With 100,000’s of devices and users accessing the service, manual management processes are highly inefficient, or even impossible to implement. Service Providers that host multiple clients on their storage infrastructure require multi-tenancy support and administration tiers. A cloud storage platform must have strong management tools, allowing for automation and scale.
Customization: Service providers in particular, but also corporate IT teams need to tailor the look & feel of a system for their end users. A cloud storage platform should offer the means to customize the look & feel of applications, interfaces, and management screens.
The CTERA Cloud Storage Platform
As a cloud storage pioneer, CTERA recognized the need for an integrated cloud storage platform to enable the delivery and management of multiple storage-based applications. Figure 2 outlines the key elements of the CTERA platform: Storage infrastructure: The CTERA storage platform
interoperates with and has been certified by leading object storage vendors and cloud service providers. Cloud Storage Connectors: Built-in interfaces to
existing storage infrastructure offerings – either object storage or file system drive. The connectors leverage industry standards (e.g. OpenStack or Amazon S3) and custom APIs to plug into any number of private or public cloud storage infrastructures. In addition to storage interoperability, the cloud storage connectors handle important back-end functionality such as load balancing between storage arrays, and mapping data to different storage tiers based on service level agreements (SLAs).
Storage Clients: The CTERA family of storage clients includes its cloud storage gateways, which are deployed in physical locations; desktop and server software agents; mobile apps for iOS and Android smartphones and tablets; 3rd party web browsers. The cloud storage gateways also offer ‘protocol conversion’ (see figure 3 next page) of local network storage protocols such as CIFS, NFS, AFP and FTP thus enabling seamless deployment of cloud storage into existing IT infrastructure. The wide range of storage clients enables the CTERA cloud storage platform to support virtually any deployment scenario, including small businesses, enterprise ROBOs, individual users, mobile users and anyone with a web browser.
connectors convert CTTP to any proprietary cloud storage protocol. This makes the integration between cloud storage and existing IT systems seamless. Security and Acceleration: The cloud storage
platform has built-in functionality for security and acceleration. Data is encrypted as soon as it leaves the storage client, ensuring full protection in transit and at rest. Users have full control over encryption keys, ensuring that their sensitive data is not accessible to others. On mobile devices, access is verified with 2-factor authentication, and data is encrypted, ‘sandboxed’ (sequestered) from other applications, with a remote wipe feature for lost or de-authorized devices. On the acceleration front, CTERA employs client side source-based de-duplication, compression and bandwidth throttling. Any data redundancy is removed before being transferred over the WAN, thus reducing network load and required cloud storage space. These techniques also enhance performance for end-users. Multi-Tenant Multi-Tier Management: Designed to
support large deployments by service providers or large enterprises, the CTERA platform offers highly scalable provisioning, monitoring and maintenance capabilities (see Figure 3). Built in templates and
automation features help administrators control 100,000’s of users and devices. There are several management tiers within the system to allow delegation of administration rights to customers, divisions, or agencies, and multi-tenancy keeps data and service plans separate between them.
Integration APIs: Designed for integration with existing IT infrastructure, the APIs enable a seamless interface with important functions such as: existing directory services (e.g. Active Directory, LDAP) for user authentication and rights management; Single Sign-On (SSO); Billing systems; and a variety of other existing IT services.
Core Applications: The CTERA cloud storage platform includes several key application workflows. Among those are: File sync & share (FSS) for secure sharing of files amongst users and devices; Hybrid backup that combines local backup with cloud backup of user workstations and servers; Team collaboration tools that support ad-hoc and project-based information sharing; Remotely managed ‘network attached storage’ (NAS) for SMBs and enterprise remote offices; etc. The platform can be extended to support future applications, developed by CTERA or its partners (through APIs).
The following table summarizes how the platform requirements outlined earlier in this white paper are addressed by the CTERA platform:
Platform requirements
CTERA capabilities
Back-end interoperability Cloud storage connectors offer certified interfaces to a variety of storage vendors and cloud storage services, configurable via a wizard interface with no coding required.
Flexible deployment The broad offering of cloud storage gateway appliances, end-point software agents and mobile apps addresses multiple deployment scenarios.
IT infrastructure integration CTERA Portal, the management system, includes APIs for billing, monitoring, and web shop front integrations. The CTERA gateways mediate between cloud storage and support local file access protocols (including CIFS, NFS, AFP). CTERA end-point agents include OS shell integration for easy, familiar operation by end-users.
Scalability The CTERA platform scales from a few to 100,000’s of users and end-point devices.
Security Data is encrypted in transit and at rest; User authentication is based on existing directory services (AD, LDAP); Mobile device data is ‘sand-boxed’ and can be wiped remotely; Single Sign-On (SSO) is supported.
Performance Source-based de-duplication, compression and bandwidth throttling ensure that users enjoy maximum performance with minimum load on network and storage resources.
Manageability CTERA Portal offers centralized provisioning, monitoring and maintenance of large scale deployments, with multiple tiers of management. Templates are employed to manage and configure a large number of remote appliances.
Customization CTERA Portal look & feel can easily be ‘white labeled’ to meet corporate or service provider branding needs.
Storage applications CTERA has multiple built-in, full-featured storage applications, such as: hybrid-backup of workstations and servers; truly private file sync & share (FSS); team collaboration services; mobile data access with BYOD support; centrally-managed NAS appliances.
Benefits of the CTERA Platform
Adopting a platform approach to cloud storage offers significant benefits: Time to market / deployment: Whether you are a
service provider looking to introduce new cloud storage services, or an enterprise wishing to utilize private cloud storage, leveraging the CTERA cloud storage platform dramatically reduces the time required to deploy the service. Service providers can gain market share and reach profitability quicker, while enterprises can realize cost savings and enjoy the benefits sooner.
Low TCO: A platform approach reduces the initial investment for deploying and integrating a cloud storage service, as well as the operational costs that follow. Most of the interoperability issues are addressed ‘out of the box’, and ongoing expenses for managing multiple storage applications accessed by a large number of users/devices are lower.
Storage infrastructure independence: The CTERA platform interoperates with virtually every storage infrastructure vendor and many public cloud storage services. This offers customers great flexibility, and helps to avoid ‘lock-in’ by a single storage vendor. Agility: Being a scalable and flexible platform, CTERA
Summary
The advantages offered by cloud storage are well understood nowadays. The elasticity, pay-per-use model and cost/performance benefits have driven broad adoption of cloud storage by service providers, enterprises and SMBs. As the technology matures, so does the deployment model. Deployment architectures that started with early adopters and a do-it-yourself approach, evolved into a plethora of ‘per-use-case’ tools, and have now reached a more mature platform stage. Service providers and enterprises who wish to deploy cloud storage and associated applications are best served with a unified cloud storage platform that handles integration, interoperability and manageability challenges.
The CTERA cloud storage platform enables enterprises and service providers to quickly deploy a full-featured cloud storage service. It seamlessly integrates with existing IT infrastructure, is architected for multi-tiered management (see figure 4), includes built-in security measures, and offers back-end storage infrastructure interoperability. With CTERA, organizations can adapt to any deployment scenario, offering robust storage applications to all users, including remote and mobile ones.
