W H I T E P A P E R
D e l i v e r i n g C u s t o m e r V a l u e w i t h E n t e r p r i s e F l a s h
D e p l o y m e n t s
Sponsored by: NetApp
Jeff Janukowicz Dan Iacono
February 2013
I N T H I S W H I T E P A P E R
In this whitepaper, IDC examines the growing use of flash technology across the enterprise and how the technology is being deployed to improve performance and efficiency. This whitepaper explores various aspects associated with deploying flash technology and discusses the importance of a full portfolio of flash solutions that is designed specifically for the enterprise.
S I T U A T I O N O V E R V I E W
The digital economy continues to grow and as it does it places an increased value on information held within our datacenters. Today, enormous amounts of data are being created from a variety of sources, such as applications, new mobile devices, big data analytics, and the cloud. This is changing the speed with which business gets conducted and the scale in which it occurs. Moreover, the digital explosion shows no sign of slowing and IDC expects the amount of data being stored to grow in excess of 50% per year over the next few years.
For the IT manager, dealing with these vast quantities of data presents a significant challenge in a time where storage budgets are limited and IT staff is shrinking. Over the last 5 years, spending on IT hardware and software has increased by almost 25%. Yet, these challenges do not fully convey the pain points that continue to exist for enterprises dealing with the data growth in a cost effective manner. Compounding the task is the need to provide higher performance and the improved responsiveness required in the most demanding environments.
Today's storage technology options are more complex than ever. To address this data explosion and extract value from it, IT managers must:
Deliver Higher Performance and Improved Responsiveness. Processor and memory performance have grown with Moore's Law. When coupled with the increased usage of multi-core processors, virtualization, and accelerated network speeds, storage performance has lagged behind. Improving the I/O data path and addressing performance bottlenecks is critical to provide the necessary performance throughput. Improving responsiveness is another critical factor. Whether accessing email, an enterprise resource planning (ERP) system, an OLTP
Lower Costs and Increase Efficiency. Hardware price per capacity costs continue to come down and the implementation of new storage technologies such as data compression and de-duplication have increased storage efficiency. However, performance improvements are not keeping up with the rate of data growth and the need to extract value from that data. This performance gap creates inefficiencies and forces enterprises to purchase ever-increasing amounts of hardware. Successful organizations focus on return on investment (ROI) and the total cost of ownership (TCO) when selecting a storage solution. Provide Enterprise Grade Service. The 24x7 nature and speed that business
gets conducted requires enterprise grade availability, reliability and flexibility features (e.g. snapshot, cloning, replication, multi-protocol access, etc). In addition, the quality-of-service (QoS) and data integrity that is expected in the enterprise challenges IT management further.
T h e E m e r g e n c e o f S o l i d S t a t e D r i v e s
Solid State Storage (SSS) in-conjunction with Solid State Drives (SSDs) is not a new technology. However, advancements in semiconductor technology and the growing use of NAND flash in the consumer market has pushed NAND flash-based SSDs into the enterprise as a cost effective solution to mitigate the performance gap by complementing existing storage infrastructures. When coupled with new software technologies, such as intelligent caching and automated storage tiering, these advances have made SSDs deployments easier and made solid state storage more usable across the enterprise.
SSDs are built leveraging a semiconductor nonvolatile memory (typically NAND flash), an advanced device controller, and an interface to connect to the host. Today, these devices are transforming the entire computing industry thanks to some of the inherent benefits:
Higher performance. Inherently SSDs can achieve multiple gigabytes per second of random data throughput. SSDs offer high input/output operations per second (IOPs) performance. For example; a single SSD can provide in excess of 10,000 IOPs – an order of magnitude improvement over the fastest hard disk drives (HDDs). Additionally, SSDs provide a more consistent I/O response time due to their fast, predictable access time and high bandwidth.
Cost savings. Solid state is a more expensive storage media when compared to HDDs on a dollar per gigabyte basis ($/GB). Yet, when intelligently integrated into a system with storage optimization technologies, storage vendors lower the acquisition cost and overall total cost of ownership (TCO). Moreover, $/IO/GB is optimized with the use of solid state storage. And the use of storage efficiency technologies such as de-duplication and compression, applied to the use of flash can dramatically reduce the effective SSD $/GB requirements.
T h e R o l e o f S o l i d S t a t e S t o r a g e i n E n t e r p r i s e A r c h i t e c t u r e s
Thanks to many of these benefits, SSDs are having a profound impact on storage system architectures. When SSDs were first introduced, they were simply used to replace existing HDDs and speed up the existing infrastructure. However, there are a multitude of different performance and capacity requirements depending on the enterprise workload. Thus, as the technology has evolved multiple approaches have emerged depending on the operating environment to capitalize on the benefits associated with solid state storage.
For the business and technology leaders, this means that flash will help transform their infrastructure by making it more physically and operationally efficient. It also means that SSD and flash technology will be utilized in multiple architectures within the datacenter to deliver on both short-term and long-term business requirements. Thus, to provide the best match for the performance and capacity necessary for the various types of workloads the following architectures have emerged:
Server-based. Lowest latency solution because the flash or SSD is closest from an I/O path perspective to the compute & application. This approach can be targeted specifically to a single application for acceleration with minimal investment. In the Network. Typically deployed as caching layer between the host and storage layers. Network caching is used for the applications when there is inadequate flash capacity at the server to achieve the required high cache hit rates.
Within the Storage Array.
Hybrid Array. Combining HDDs with SSDs in conjunction with intelligent data placement software or policies. In these solutions, SSDs can be leveraged as either persistent storage (written to the drive and can survive a power cycle) via automated tiering technology or as a cache layer within the array. In either case, a relatively small amount of NAND flash is used to accelerate and optimize the system's performance.
F U T U R E O U T L O O K
Flash will have a profound impact on the future of enterprise storage. Memory-based storage systems and SSDs have been deployed in enterprises for many years in environments that demanded the best performance and regardless of the cost. However, thanks to the declining cost of NAND flash media and system level advancements, IDC believes that SSDs and solid state technology will become pervasive across the enterprise and complement existing storage systems. Figure 1 highlights the growth of flash solid state storage.
F I G U R E 1
W o r l d w i d e S o l i d S t a t e S t o r a g e C a p a c i t y S h i p p e d
Source: IDC, 2013
B r i n g i n g F l a s h t o t h e F o r e f r o n t
For enterprise customers, flash storage changes everything. It permanently transforms the speed of business and it makes "real-time" an expectation, not an aspiration. However, at the same time, enterprise customer imperatives remain the same:
Performance
Cost and storage efficiency
Service including enterprise grade availability, reliability and flexibility features and quality of service (QoS)
Today, business and technology leaders need a scalable approach that delivers on both their short-term and long-term business requirements. When deploying flash "one size" does not fit all and a full portfolio of flash solutions that is designed specifically for the enterprise to match customer needs is imperative.
For IT managers considering flash, gone are the days when flash was only used in the high performing environments that could justify the cost premium. Yet, integrating SSD haphazardly is not the answer either. A framework is necessary to help customers determine the optimal Flash solution for their specific needs. For example: Are you currently over-provisioning storage capacity to achieve your performance
objectives?
Does everything need to be faster or just specific datasets or volumes?
What types of workloads will be targeted for performance improvement with flash?
Are you trying to make an application faster in terms of higher IOPS or want to achieve consistent latency?
What types of workloads will be used?
What is the scale of your environment today? What are the projected scalability requirements over the next 1-2 years?
How will Flash integrate into your existing platform?
Will factoring in the improved space utilization and lower power/cooling costs change the TCO calculation?
T h e F u t u r e o f F l a s h A r r a y
An emerging option for many organizations is the all flash array. The all flash array storage market has been and continues to evolve as adoption and diversity of use cases increase. IDC is observing a bifurcation in the all flash storage array market as segment begin to take shape:
Absolute Performance. There is a need for extreme raw performance of throughput, IO, and latency. In this use case, performance is paramount, therefore anything affecting performance should not be included in the storage array data path such as virtualization, data efficiency, or enterprise features (i.e. snapshot, clones, replication).
Enterprise. Acceleration of applications is important and will provide consistent latency faster than traditional spinning disk, but enterprise feature sets such as snapshots, clones, site replication, and data efficiency must be included to ensure data protection.
N e t A p p ' s A p p r o a c h t o F l a s h
For NetApp, SSDs and flash are core strategic technologies. NetApp was one of the early implementers of flash and SSD into their product portfolio and has a long history of accelerating applications with flash. NetApp's current flash portfolio extends across the server, network, and storage controller to the array. The company has already shipped in excess of 36 PB of flash to its customers. Its initial focus was on performance and efficiency driven by intelligent caching (Virtual Storage Tier - VST). VST, which provides end-to-end intelligent caching for Data ONTAP based storage systems, supports real-time assessment of workloads based priorities, and enables I/O data requests to be optimized for cost and performance without requiring complex data classification. As part of the Data ONTAP operating system, VST achieves efficiency through data deduplication and thin cloning integration Technologies supported includes both PCIe and solid-state disk (SSDs).
The core technologies within the NetApp VST are Flash Cache, Flash Pool and Flash Accel.
NetApp Flash Cache controller-attached PCIe intelligent caching, speeds data access through intelligent caching of recently read user data or NetApp metadata. Flash Cache works with either SAS or SATA drives in a hybrid configuration.
NetApp Flash Pool provides caching of both random read and write operations through the automated use of SSD drives, enabling the use of capacity optimized HDD technology across the majority of application workloads. Flash Pool enables the creation of a RAID protected Data ONTAP aggregate that is comprised of a combination of HDDs and SSDs.
NetApp Flash Accel software combines the capabilities of server-based flash technology and NetApp intelligent caching to extend VST to the server level. Flash Accel software supports any server based PCI-e Flash card or SSD drive including Fusion I/O to ensure that the data remain coherent, or consistent, with the back-end storage.
Value of VST to Shared Storage Infrastructure and Broad Range of Enterprise Apps
Having flash in a shared storage environment provides ability for multiple applications to leverage the benefits SSD acceleration. Key value propositions for VST include: Real-Time promotion of hot data.
Integration of key storage efficiency technologies including deduplication, thin provisioning and SATA technology.
Simplicity of management with no requirements for policy changes or data reorganization.
Examples of uses cases which can benefit from VST include:
Database. Have a tendency to favor reads versus write operations (with a ratio of 8:1 as a rule of thumb).
Big Data. Accelerating results of analytics beyond the capabilities of HDD spindles alone.
NFS Applications. Intelligent caching of the data and metadata accelerates NFS operations.
N e t A p p C u s t o m e r E x p e r i e n c e s
IDC spoke with two customers identified by NetApp who have deployed flash technology solutions to support a wide variety of applications. Below is a summary of each interview, highlighting the benefits of deployment and the relationship with NetApp.
Indian Hills Community College
Indian Hills Community College (IHCC), located in Ottumwa, Iowa is a two-year college with a student population of nearly 4,000. IHCC is hosting in excess of 600 VDI desktops and running multiple production applications simultaneously on a single NetApp 3240. Utilizing Flash Cache, the school claims to have achieved average latencies less than 2ms.
In January, 2013 IDC spoke with IHCC CIO Shane Molyneux regarding the NetApp deployment. Mr. Molyneux stated that NetApp was chosen for the school's next generation storage solution based largely on virtual desktop integration. As he explained, "The primary driver was around a virtual desktop environment, and we needed a way to deploy and get a large number of virtual desktops without having boot-storm issues."
IHCC did not have any flash deployed before the NetApp solution, but Mr. Molyneux knew he needed a low latency solution to, "make sure we didn't ever have any issues with lagging clients. It's been rock solid since we installed it." Performance wasn't the only criteria for choosing the NetApp solution. It was also enterprise data efficiency features (e.g.; data de-duplication) where, Mr. Molyneux's "conservative estimate was that it would save 40% to 50% of space and when we actually finally got it implemented, it was more like 80%, that's how much space we saved just by using de-dupe alone."
Excanto AB
IDC interviewed Tomas Rundberg, Head of Excanto's Datacenter, in January 2013. Mr. Rundberg assessed NetApp's main business value to the company as follows: "Using flash, Excanto can deliver more service on the same storage volume. That's the most important thing for us because with the flash technology you can get much more IOPS without buying extra storage volume that you wouldn't use."
Mr. Rundberg also notes the importance of operational efficiency that NetApp creates for Excanto, "The cache functions that we have, minimize the data administration. It is simply plug-and-play." Looking towards the future Mr. Rundberg states that Excanto plans to extend its storage strategy with NetApp by creating a "SSD storage array for super performance storage to the market. Always be as fast as possible, not only to recent used data as flash technologies provide."
N e t A p p ' s A d o p t i o n o f F l a s h B e y o n d
D a t a O N T A P t o A d d r e s s R e q u i r e m e n t s o f H i g h P e r f o r m a n c e D e d i c a t e d W o r k l o a d s
There is a need in the market for absolute performance with regard to throughput, IO, and latency beyond what traditional spinning disk or hybrid flash solutions can provide. To address the need for absolute performance in conjunction with enterprise class storage capabilities, NetApp has released the EF540.
Value of EF540 Flash Array
The NetApp E-Series storage systems are designed to meet demanding performance of data intensive workloads without sacrificing enterprise features or consistent performance. Building upon 20 years of storage experience with over 500,000 E-series delivered to customers to date, NetApp has delivered the EF540 flash storage array that can meet absolute performance needs, consistent latency and provide enterprise features that organizations depend on. Through a single 2U unit, the EF540 provides up to 19.2TB of raw capacity, while achieving 6 GB/s throughput and 300,000 IOPS sustained at <1ms latency. Beyond sustained performance, the EF540 delivers enterprise features to the all flash storage array such as:
Non-disruptive Firmware upgrades
Online configuration, expansion and maintenance Redundant I/O paths
Snapshots
Clones with volume copy
E x t e n s i o n o f N e t A p p V a l u e w i t h N e w B u i l d f r o m B o t t o m U p F l a s h A r r a y F a m i l y
( F l a s h R a y )
As discussed earlier, to bring all flash arrays to the forefront, enterprise customers require more than just performance. Storage efficiency, enterprise class storage capabilities, scalability and QoS are the critical requirements in conjunction with performance to drive broader adoption.
Throughout the past two decades, NetApp engineering has stood the test of time, which can be best viewed in the longevity of the WAFL file systems and the robustness of the E-series. There is a new paradigm shift in the market where the following three fundamentals have changed:
Performance. Scaling-out out solutions will be the preferred method to grow a storage capacity and performance with low latency.
Cost. Data efficiency techniques will be a function included in primary storage. Service. Evolution from a technology centric to a service centric IT organization
with cloud attributes.
Leading the way to the new storage paradigm is NetApp's EF540 and a new all flash storage array, FlashRay, focused on enterprise features to meet today's needs, while laying the foundation for the future of NetApp's storage. NetApp chose to build FlashRay's operating system in-house from the ground up on top of commodity components to leverage NetApp's 20 years of software engineering heritage:
NetApp's all flash storage array FlashRay will address the storage paradigm shift in three essential areas:
Performance. FlashRay is designed to provide high IOPS, high throughput with consistent sub 0.5ms latency. In addition, its scale-out architecture will help deliver the performance scaling.
Cost. Inline de-duplication & compression along with use of optimized data placement algorithms help FlashRay reduce the cost associated with expensive flash media while maximizing the flash life.
Service. FlashRay has enterprise grade availability, reliability and flexibility features (e.g. snapshot, cloning, replication, multi-protocol access, etc). In addition, automated orchestration and quality-of-service (QoS) capabilities would help enablement of service centric IT organization.
C H A L L E N G E S / O P P O R T U N I T I E S
Over the last 50 years, end users have become comfortable with the performance characteristics, management, and handling of HDDs and HDD-based storage systems. When SSDs were first introduced a number of years there were some concerns about the technology, which is natural for a newer technology. While many inherent benefits of the technology are clear, flash does behave differently than traditional HDDs. Thus, flash technology's performance and endurance characteristics need to be optimized for enterprise environments which is the reason working with a trusted vendor is vital when developing a framework that leverages flash.
Cost remains a key hurdle for many IT managers evaluating the technology. Flash is a more expensive storage media compared to HDDs on a dollar per gigabyte basis ($/GB). Yet, with proper integration enterprise-grade Flash can deliver on both their short-term and long-term business requirements by improving storage efficiency, reducing overall storage environment costs, and accelerating business critical application performance.
In addition to performance and storage efficiency, flash solutions need to provide features, such as, availability, reliability, manageability that are expected from enterprise storage systems. Unless these enterprise-class capabilities are provided, the broad industry adoption of flash would remain a distant possibility. This is particularly true for all flash arrays where the performance, to large extent, and storage efficiency, to some extent, has been the primary focus and enterprise capabilities have been largely ignored so far.
Another hurdle for all flash storage arrays is scalability. The trend is to have fixed amounts of memory and storage processing per storage capacity limiting the ability for scale-up, which causes IT managers to scale-out for growth. Scaling beyond a single unit as a pool of shared and redundant storage resources is a difficult task, especially at the performance and latency expectations of flash storage device. It will be important for the industry to address scale beyond a single appliance that increases performance linearly without compromising performance.
C O N C L U S I O N
Flash will have a profound impact on the future of enterprise storage and system architectures. The benefits of the technology make the integration of flash an ideal choice of storage media to deliver the performance and efficiency in today's digital economy. When integrated with enterprise reliability and scale, flash storage is poised to meet the requirements of the most demanding infrastructures.
C o p y r i g h t N o t i c e
External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason.
