Flash In The Enterprise

(1)

Flash In The

Enterprise

Technology and Market Overview

Chris M Evans, Langton Blue Ltd

(2)

The Need for Flash Storage

Flash storage has become one of the most talked about technologies of the last 12 months and has moved from being a niche product to a mainstream consideration for storage in the Enterprise. This white paper discusses flash technology, what it is, why it should be used and how to use it.

Since the mid 1950’s, storage deployments have always used spinning disks - hard disk drives (HDDs) as the main permanent storage medium. HDDs consist of one or more magnetic platters (or disks) accessed by a mechanical read/write head that moves across the surface of the platter, accessing data on concentric tracks. Although hard drive manufacturers have made great gains in increasing the areal or bit density of hard drives, managing to store ever increasing amounts of data within the same physical form factor, gains in performance have not kept pace with capacity.

IOPS Density

The relationship between performance and capacity can be quantified by a measurement called IOPS density. This is the ratio of input/output operations (I/Os) per second per unit of capacity, typically expressed as gigabytes or terabytes. Application IOPS requirements have increased over time as the performance of processors, memory and I/O backplanes has improved.

The I/O Blender

As server and desktop virtualisation becomes more prevalent, the workload profile of I/O on shared storage becomes much more unpredictable and as a result today’s workload is highly randomised. This presents a particular problem for hard drives that have to mechanically move the write head into position and wait for the drive to rotate to the right position before data can be read. At worst case, each I/O could be random enough to include full rotational delay (waiting for the drive to turn) and seek time (moving the head to the right position), in total a value known as latency.

Flash to The Rescue

(4)

What is Flash?

Flash storage is solid-state non-volatile memory, similar to that used in consumer devices, such as iPods and USB memory sticks. The contents of flash storage are not lost when power is removed from the device, making it a permanent storage medium. Today’s flash devices are based around NAND logic technology and packaged with controller logic circuitry that manages the process of reading and writing from NAND enabling them to be suitable for enterprise-class environments.

Flash has a number of benefits over traditional spinning media.

• Performance – flash devices deliver high performance, both in terms of throughput and latency, even for random I/O.

• Environmental – flash devices are lighter and consume much less power than their hard drive counterparts. They create no vibration and require less cooling. Although solid state drives are packaged to look like hard drives, in reality they can be made much smaller than the typical hard drive form factor and so have a high space density (capacity to space occupied) characteristic.

Flash Memory Technology

NAND Flash memory was originally implemented using a technology known as SLC or Single Level Cell. This stores one bit of data in each memory cell. SLC is expensive but more resilient than other forms. MLC or Multi-level Cell records multiple (typically 2) bits of information per cell. Both MLC and SLC technology have finite lifetimes, being capable of only a fixed number of write operations. SLC lasts longer than MLC, as it only has to store two possible values or states. Some vendors have introduced MLC technology using three or four bits per cell, by storing up to eight or sixteen states respectively and this is being used to create high capacity devices.

Managing NAND

Flash memory has a finite lifespan, limited by the number of times data can be written to the device. Data is written to a device in blocks (typically 4KB) and once the contents of a block are set it must be erased before it can be written again. Flash devices have a fixed number of erase cycles, usually around 100,000 for SLC and 10,000 for MLC devices. As a result, without some management, flash devices would have a short lifetime and be very unreliable. The flash storage vendors (those selling SSDs and arrays) use a range of management techniques in order to prolong the life of flash, including wear levelling, overprovisioning, write amplification and garbage collection.

Flash Form Factors and Packages

Flash can be packaged in a number of different ways. Today we see the following implementations of flash storage.

• Solid State Disk (SSD) – these devices package flash in the form factor of a traditional hard disk and use existing disk-based protocols for connectivity, including SATA and SAS. SSDs have on-board controllers that both manage the interface with the host and the flash itself and implement management algorithms. Performance of SSDs is usually measured in sub-millisecond figures.

(5)

access protocol in place, PCIe SSD devices usually require some host driver software and this can consume processor cycles and memory. PCIe performance is typically measured in tens of microseconds.

• NVDRAM – this is a server memory slot or DIMM form factor device that uses the existing on-board memory slots normally reserved for DRAM on the server motheron-board. By plugging directly onto the memory bus, NVDRAM delivers exceptionally high performance, typically measured around five to ten microseconds. The technology looks like DRAM and so BIOS changes and O/S drivers are required to differentiate the media as solid state and not volatile memory.

(6)

Deployment Scenarios

Flash technology can be deployed in a number of places within the data centre, including in the server itself, or in a centralised storage array or appliance.

Server Deployment

Flash technology can be deployed directly in the server, with three form-factor choices currently available - SSDs, PCIe SSD adaptors and NVDIMM (non-volatile DIMM) modules.

Solid-state disks can be used simply as replacement for traditional hard drives and will boost performance by improving throughput and lowering latency. However when only used as a tier of storage, SSDs may be under utilised and of course not managed effectively to ensure optimum lifetime. SSDs can also be used as cache devices, speeding up I/O to and from traditional hard drives. These solutions need additional caching software, either as an add-on driver or software product or built into the operating system.

PCIe SSD devices can be used either as cache or primary storage devices. Some PCIe SSD drives are automatically recognised as disk drives but most require additional software in form of drivers or software that integrate with the operating system and cache active data to improve performance. NVDIMM devices are now available from a number of vendors, most notably IBM in the form of their eXFlash offering. NVDIMMs have similar characteristics to PCIe SSD in that they bypass the storage I/O bus but are even closer to the processor backplane and have very low latency.

Deployment of flash directly into the server removes the I/O overhead of the storage area network (SAN), lowering latency and improving throughput and performance. However directly attached devices are not shared and isolated to access only by that server itself. This causes problems if the server crashes or fails due to a hardware error. It also requires planning to cater for redundancy of the data if the server copy is the only copy available. Consequently many vendors offer solutions that act only as a read cache.

Caching Options

A cache stores copies of the most frequently accessed data in a server in order to improve I/O performance, however the cache copy is not the primary copy of data. Cache solutions can be used to improve either read or write performance or both. Read performance is improved by retaining frequently accessed blocks of data in cache and servicing I/O requests directly from the cache copy. If a block of data is updated, the cache is written with the new block and also stored on external disk, a process known as write-through caching.

(7)

Shared Storage

Storage arrays, deployed as part of a storage area network or directly attached to a server can be ameliorated with flash devices. The initial use of flash in recent years has been in adding SSDs to traditional disk arrays. This provided some performance improvement but typically didn’t make best use of the flash resources as the underlying architecture of these arrays was designed to cope with slow permanent storage media. As a result the implementations were only a stepping-stone to better devices.

As the use of flash in arrays evolved, we have seen a split into three distinct categories of device, the flash-optimised traditional array, the flash-enhanced array and the all-flash array.

Flash Optimised

These arrays are based on traditional storage array devices but have been amended to cope with flash as the main storage media. Amendments include redesigning the array software to cope with flash characteristics and adding extra processing and memory capacity. Regardless of the amendments, the hardware itself still accesses the devices using the same interface as used with hard disk drives.

Flash Enhanced

These devices use flash to improve the performance of the array, through a variety of software-based techniques and new architectures. For example, the ZFS file system is implemented as the underlying storage file-system layer and this caches all writes to flash before committing asynchronously to disk. These devices are also known as hybrid arrays as they combine flash and hard drives together for a better price/performance ratio.

All Flash

These devices have been designed, architected and built to specifically work with only flash storage media. Their design intrinsically caters for the characteristics of flash devices and includes flash media delivered as solid-state disks, PCIe SSD or custom flash modules. The all-flash array category represents the fastest of devices, includes a greater set of space management features such as compression and de-duplication and are typically architected for long service life.

Requirements

Whatever category of product is chosen, the underlying requirements should be considered. The following table highlights some of the considerations when choosing a specific platform.

Characteristic Flash Optimised Flash Enhanced All-Flash

Performance Good, but can be hindered by the underlying HDD-based architecture and needs software features to ensure flash is used effectively.

Good, but can suffer performance issues on read I/O if data is not available through the cache.

Excellent, architecture designed for high performance and scalability.

Cost Good, flash can be added as a complement to HDDs to target performance but needs

Acceptable; costs are contained by using flash in a limited fashion.

High. These devices are targeted at high performance workloads and flash is still relatively expensive. Some vendors are using de-duplication to reduce per GB costs.

Features Good, typically all the mature features of traditional arrays are available for flash disks.

Good, hybrid solutions usually

support advanced features. Acceptable. Most manufacturers have not supported advanced features like replication and snapshots.

Scalability Good, these arrays are based on traditional technology, however they may not be capable of scaling fully in all-flash designs.

Good, scaling is achieved through capacity with hard drives and through performance with flash.

(8)

Justifying The Use of Flash Technology

It is clearly obvious that flash storage can add significant performance improvements over traditional storage media. However, flash technology is more expensive than hard drive-based storage when compared purely on a $/GB basis and this can make justification for using the technology more difficult. Here are some thoughts on developing a flash deployment strategy.

• Identify requirements. Without a clear idea of either the business benefit delivered or problem solved, justification is almost impossible. This means a clear set of requirements need to be established with details on exactly where within the infrastructure flash would be used. • Determine how much flash is needed. It’s easy and tempting to think all-flash arrays are

the solution to all requirements, however these devices are expensive and may provide way more throughput than can be utilised. By evaluating the performance improvement needed as part of requirements, the specific need for flash can be pinpointed, making it easier to determine whether server or array-based solutions are best.

• Consider $/IOPS. The traditional way of measuring storage cost is to look at the “per TB” or “per GB” price. This method was useful when storage was delivered as a single tier and became more complex with multi-tier solutions. The benefits from flash are not delivered in capacity terms but by throughput and latency, therefore it makes sense to use this metric to calculate the cost/benefit of implementing flash compared to traditional hard drives.

• Calculate business benefit or loss. Implementing flash may provide a number of quantifiable benefits, including higher transaction/order throughput, offsetting additional server or array purchases or significant savings in environmental charges. These can all be built into a business case.

• Evaluate. Flash is a new technology and the market is full of mature solutions in each category. Evaluating specific vendor offerings can provide more detail and knowledge on how flash will best work in a particular environment and validate whether vendor claims stand up to scrutiny.

• Use your vendor. There are a large number of vendors, both incumbent and start-up, looking for business. They have plenty of collateral in the form of white papers and guides that help justify the use of their products.

• Use Langton Blue. Vendors have great material and place their spin on the industry. Langton Blue provides independent advice without vendor hype.

(9)

More Information

The full version of this report is available on request, containing additional technical background and vendor reviews. Contact [email protected] for more information and pricing details.

Langton Blue Ltd is hardware and software independent, working for the business value to the end customer. Contact us to discuss how we can help you bank the benefits in your technology investments. Website: www.langtonblue.com Email: [email protected] Twitter: @langtonblue Phone: 0845 275 7085 Post: Langton Blue Ltd 78 York Street London W1H 1DP

The Author

Chris M Evans has worked in the technology industry since 1987, starting as a systems programmer on the IBM mainframe platform, while retaining an interest in storage. After working abroad, he co-founded an Internet-based music distribution company during the .com era, returning to consultancy in the new millennium. In 2009 he co-founded Langton Blue Ltd (www.langtonblue.com), a boutique consultancy firm focused on delivering business benefit through efficient technology deployments. Chris writes a popular blog at http://blog.architecting.it, attends many conferences and invitation-only events and can be found providing regular industry contributions through Twitter (@chrismevans) and other social media outlets

No guarantees or warranties are provided regarding the accuracy, reliability or usability of any information contained within this document and readers are recommended to validate any statements or other representations made for validity.