SPECIAL ISSUE
for DataCore Software
provided by
nt4admins.de
Editorial
This month's topic is storage virtualization with a focus on Virtual SAN – This year VMware launched a vSphere-based virtual SAN for converged server and storage systems, Nutanix has released version 4.0 of its pre-packaged hardware and software appliance offering and DataCore released SANsymphony-V10 storage virtualization software and with it, a new virtual SAN capability designed to work with any hypervisor or storage. We therefore decided to compare these different approaches and concepts. Please note, we utilized relevant information from a related article that appeared within our last E-Paper, portions of that material, including the concept behind VMware’s VSAN has been reused here to
Virtual SAN – DataCore versus VMware
DataCore’s Virtual SAN or VMware’s Virtual SAN – this type of comparison and
quest-ion easily comes to mind whenever the use of storage virtualizatquest-ion is being evaluated.
Other important issues to consider is the ease of deployment and ability to scale these
systems in terms of performance and capacity, a key point highlighted by converged
system suppliers such as Nutanix for example. With this backdrop and the introduction
of DataCore’s SANsymphony-V10, and the recent release of VMware’s Virtual SAN,
we decided there was a need to offer a comparative analysis overviewing the “virtual
SANs” and their essential differences.
A major upheaval is taking place in the storage arena – the growing relevance of software and the agility it brings are clearly gaining momentum. The current buzzword of note in the industry is software-defined storage (SDS) and it is causing a fundamental change on how we implement storage infra-structures. To highlight the point, storage specialist EMC last year introduced its VIPR technology, a software-based approach that it claimed was designed to address and in-tegrate lower cost "commodity hardware".
This endorsement of a software based ap-proach running on ‘commodity hardware’ has led many users to rethink their storage approach. Following the maxim: ‘If the lea-ding storage vendor EMC says that storage is a software subject, then it must be ok to start using software solutions that operate on low-cost standardized hardware’. This along with all the momentum around SDS over the last year has caused a major dis-ruption in the mindset on how to approach storage going forward.
With the general acceptance of flash-based storage and within application host servers, a further issue comes into play. All the major flash memory manufacturers are trying to
develop their solutions for the storage mar-ket and therefore they now have to acquire or write their own software to manage and integrate these new high-speed solid state storage devices into their solution offering. They need more than hardware, they need to deliver a complete ‘storage software stack’ to effectively compete. This is due to the fact that even with flash-based storage sys-tems, the user wants to be abstracted from the details and demands functionality such as snapshot and the ability to tier storage
between flash and existing disk based sto-rage investments. Data mirroring to protect data must also be supported. Likewise, it is expected that there will be solutions for metro-clusters and remote distance replica-tion – all these funcreplica-tions are essential and now expected. In addition, integration with new and existing virtualization platforms presents providers of flash memory devices with a number of challenges. Moreover, the incorporation of flash storage alongside the use of local hard disks within virtualized host systems running applications on commodity servers is proving to be a very cost-effective option for providing high-performance sto-rage services.
However, before less expensive shared sto-rage approaches are made fit for use in mission-critical applications – particularly in relation to high-availability – there are a number of developments that need to be taken into account. VMware is attempting to take advantage of this trend and there-fore introduced a virtual SAN (VSAN) that has been designed to rely on key vSphere vir-tualization platform services to provide core functionality especially in terms of ensuring higher availability.
This concept was announced last year and at that time a beta phase was launched that lasted over a year. Earlier this year, version 1.0 of VMware’s VSAN was finally released. Interestingly, with VSAN, VMware has not set the bar particularly high: VMware claims VSAN can do 2 million IOPS and a maximum 4.4 petabytes of capacity in a 32 node VSAN cluster. They also clearly positioned the solu-tion for smaller business and less demanding application use cases.
VMware VSAN Overview
With VMware, VSAN wanted to open the door for small and medium-sized (SME) enterprises. VMware plans to enable the-se SME customers to virtualize their critical applications and to have access to highly-available low-cost local storage. The VSAN system requires flash and magnetic disks to be installed directly within the servers. The magnetic disks provide the storage capacity while the required flash devices (i.e. solid sta-te disks) are responsible for caching, hence yielding higher performance. In principle, the VSAN software combines the storage running on various ESXi hosts – i.e. the vir-tualization hosts – into a distributed pool of shared storage. The flash drives are there only to assume the role and function of read
and write cache memory accelerators and do not contribute to the overall storage capacity of the cluster.
When a new virtual machine (VM) is deplo-yed, the virtual disks (also known as VDMKs) are assembled from the common pool. The virtual disks are thus provided with different performance and availability service levels, depending on the relevant profiles assigned to the VMs. If a vSphere cluster uses hosts which are deployed only as compute nodes (i.e. they do not contribute storage capacity to the cluster), then these systems can utilize storage from the distributed pool (this is also known as a datastore).
VMware’s VSAN depends heavily on its use and integration with other vSphere functions and services. The VSAN software mirrors the VMDKs onto one or more physical servers and in this way it avoids ‘single points of failure’ scenarios. The VMs in a cluster can be moved from one physical host to another – to achieve this, VSAN interacts with the vMotion and distributed resource scheduling (DRS) functions within vCenter. The system does not require its own external storage arrays or dedicated file servers. Cache con-tents may also be migrated to ensure mini-mal performance impact if the VMs continue to work at the new location.
DataCore’s Virtual SAN
Overview
With SANsymphony-V, DataCore’s Virtual SAN – one of the earliest established soft-ware vendors in the field of storage virtua-lization – has launched an approach which is suitable not only for virtualized environ-ments like vSphere, but also for other
hy-pervisors such as Microsoft Hyper-V, Citrix XenServer, or any other hypervisor capable of running a Windows virtual machine. As with VMware's VSAN, DataCore’s SANsym-phony-V10 (the 10th version) fully utilizes and transforms local storage resources from the application servers (SAS, SATA, SSD, etc.) into shared storage pools. However, unlike VMware VSAN, DataCore virtual SANs can also manage and integrate the resources from both dedicated traditional SANs and directly attached storage (DAS) systems, or even cloud-based storage solutions.
In SANsymphony-V10, a number of I/O performance tuning-related features have also been added. The number of nodes in a SANsymphony-V10 server group has been doubled to 32 nodes. Additional network technologies such as 40/56 GigE iSCSI and 16 Gbit/s fibre channel are now also suppor-ted. The iSCSI NIC teaming feature is sup-ported. The result of these enhancements is drastically improved storage performance. SANsymphony-V10 can be used to build
virtual SAN environments which achieve throughputs of 50 million IOPS and support up to 32 petabytes of capacity.
With the help of special tools such as heat maps for performance visualization, the sys-tem administrator can quickly see the real-time placement of hot and cold data on the corresponding storage pool disks. Additio-nally, automatic migration and distribution of data on the appropriate storage devices (with frequently used data going to the faster drives) – known as automated tiering – has been further improved, to allow better ex-ploitation of more expensive resources such as flash storage. The redistribution of data, for example, if individual hard drives need to be removed from the storage pools, is auto-matic. No downtime is necessary. DataCore has also improved its caching algorithms to obtain even higher performance out of the latest server technology. This is designed to allow better use of flash; especially in terms of write access which is key in many data-base and transactional workloads (i.e. ERP, SAP, Oracle, Microsoft SQL, Exchange and SharePoint).
DataCore’s virtual SAN provides a compre-hensive feature set. However, when there are many features available, installation can be-come generally quite complex. SANsympho-ny-V10 has made major improvements in this area. A Smart Deployment Wizard offers templates for various deployment options, such as virtual SANs, highly available SANs, or NAS file servers. The desired configuration can then be started with a single click of the template. The templates give system admi-nistrators a clearly defined starting point and guide them through deployment scenarios, from which they then can add further func-tions when they are needed.
Picture 2. Autotiering provides up to 16 tiers for data. Source: DataCore
Picture 3. DataCore easily integrates Flash based memory and supports caching algorithms to optimize and accelerate IO performance. Source: DataCore
VMware VSAN and DataCore
Virtual SAN Differences
When comparing DataCore’s SANsympho-ny-V10 Virtual SAN with VMware’s VSAN, one aspect is particularly striking: VSAN is suitable only for ESXi-based hosts which have a directly connected storage device – hard drives and SSDs/flash memory devices. By contrast, SANsymphony-V is suitable both for physical environments and virtualized in-frastructures. It also supports other Hypervi-sor environments, such as Hyper-V, Xenser-ver, etc. This means that storage functions from "normal" systems and existing storage investments are also available and usable – i.e. non-virtualized environments that run on Windows, Linux, Unix or MacOS operating systems. A further advantage of DataCore’s Virtual SAN as compared to VMware’s VSAN is the ability to take advantage of locally connected storage, and also external sto-rage systems and traditional SANs that are connected to the server via the standard in-terfaces (ie Ethernet, Fibre channel). This ability to utilize universal storage servi-ces and features that span the entire infra-structure makes managing the overall sto-rage environment really simple for the users. In SANsymphony-V10 all actions –
provisi-oning, allocating, protecting and managing storage – can be carried out from a single console. This means that the administrator does not have to use different vendor tools for various Flash, storage arrays and disk devices which minimizes the learning cur-ve – especially in cur-very heterogeneous multi-vendor storage environments. Also worth noting, the VMware vCenter, VAAI and Microsoft System Center plug-ins are made available to simplify storage management and hypervisor integration.
Performance. The issue of data throughput is a major differentiator. In VSAN the specified read and write speeds are achieved by the use of SSD storage as a buffer. This is different for SANsymphony-V10, where the buffers are held in working memory – i.e. in the DRAM. Moreover, access to the DRAM is many times faster than access to flash-based memory. Therefore, with SANsymphony-V, the use of flash-based storage systems are not absolu-tely necessary to be used in order to achie-ve a high data throughput (achieving oachie-ver 1 million IOPS per node is common and practi-cal). The realized performance of DataCore’s Virtual SAN is significantly higher than the throughput of VMware’s VSAN. As a result, the solution is less expensive than VSAN while also delivering greater performance.
Picture 4. SANsymphony-V combines virtual and physical SAN environments. Source: DataCore
In VSAN, replicated write accesses to flash or SSD memory must be "committed" on different hosts before confirmation of the transaction is delivered back to the appli-cations. This is done differently with SAN-symphony-V10: here a faster confirmation is obtained, since it is based on "memory to memory mirroring" technology. In addi-tion, IT professionals should certainly allow for the possibility of a cache hit. If such a hit occurs during read access, SANsymphony-V10 fetches the data blocks directly from the DRAM. In this way, the system does not have to wait for the flash memory or even the hard drives.
Caching of the data in the working memory (DRAM) has another positive effect: there will be fewer read-write cycles that need to be performed directly on the SSD media. This both improves the service life of the ex-pensive flash memory devices and there are fewer of the delays which typically occur in flash memory when older blocks need to be overwritten. As far as high availability (HA) is concerned, SANsymphony-V10 works according to the "N+1" principle. This means that the software can be applied even on a single node – but then there is no failure resilience. High availability is only available with an additional node.
In contrast to DataCore, all the "converged storage" approaches, including VMware’s VSAN, need at least two nodes plus one as a "witness" node (also sometimes known as a quorum node). This means at least one ad-ditional node is required, in order to ensure high availability which adds cost and com-plexity. This is obviously not as efficient as the "N+1" approach where the information can be distributed over all nodes (no special purpose ‘witness nodes’ are needed) – in SANsymphony-V10 for example this only re-quires 2 nodes and can scale to a maximum of 32 nodes currently. Another advantage of DataCore’s HA approach in direct
com-parison to the converged approach, is that DataCore offers a "metro-wide mirroring" facility. The software can run on a VM or on a server and the 2 instances can be located in 2 different racks, 2 different rooms or across town at 2 different sites. This brings further benefits, since a complete location can be mirrored – and over greater distances. If multiple nodes are used in SANsymphony-V10, the write access procedure is as fol-lows: the system writes to the cache memory (DRAM) on one node then writes to cache located on a second node. In high-availability configurations, these two nodes are usually located at two different locations, then a
wri-te-back to the memory takes place, a confir-mation is made and everything is completely confirmed, this insures that the information is located in 2 locations and avoids the sing-le point of failure scenario. This is the most efficient design and represents the design used in all the high-performance enterprise-class controllers and arrays in the storage industry. This basic approach also called a ‘2 stage commit’ is what EMC, IBM, Hitachi and others have implemented and it well proven in the real world.
Another design concept in modern storage architectures remains to be considered. In VMware's Virtual SAN, storage virtualization
Comparison with Virtual SAN
With the VSAN concept, VMware is tar-geting a very specific clientele. These are the companies which do not want to use expensive, stand-alone storage arrays, but would prefer to combine multiple servers with directly attached storage devices to form a convergent system. Many small and medium-sized enterprises don't want to incur the costs associated with purchasing and operating dedicated storage arrays or complex traditional SANs. Each storage system uses its own proprietary tools to operate storage and to manage functiona-lity within the array. If one owns different models or brands then each has specialized training and support requirements as well, the management of these systems quite quickly becomes very complicated.
Generally speaking, it therefore seems that VMware with its current offering has inspi-red but avoided engaging with larger-scale competition and with major storage manu-facturers such as EMC, Netapp, HP, IBM or Fujitsu. Note:
• There is currently a maximum of 32 ESXi-based hosts allowed in a cluster delivering relatively modest performance of 2.milli-on IOPs
• In general at least three hosts (each with at least one hard drive and a SSD) are re-quired in a cluster.
• Each host that wants to add its rotating disks to the VSAN pool must also have the added expense of a flash memory / SSD device.
• The Flash or SSD memory devices are only used as read-write cache memory.
• Only devices connected locally via the SATA, SAS or PCIe interfaces are suppor-ted.
• Only ESXi-based hosts within the VSAN cluster can use the storage thus created. The maximum size of a VMDK (i.e. a virtual hard disk in a VSAN datastore) is limited to being not more than 2 TByte. With ESXi, a maximum size of 62 TByte for a VMDK is currently supported, but only on a "non-VSAN Storage device".
Only one VMware datastore with VSAN sto-rage can be created. One VSAN datastore is created per VSAN cluster; this can then be "sub-divided" into up to five disk groups in order to optimize the failure resilience; each disk group consists of one SSD and at least one or up to 7 hard disks. The exact "per VM availability" is defined by the policy ma-nagement (VM-centric). Within a vCenter, however, multiple clusters and hence multi-ple VSAN clusters can be created. Migration of VMs between these VSAN datastores is possible.
If an ESXi host needs to be transferred to Maintenance Mode, this will take quite a long time to set up with VSAN. VMware re-commends that you perform a "full migra-tion" of the VSAN data onto the remaining ESXi hosts within a VSAN cluster before switching to "Maintenance Mode". There are also other, faster options, but with the-se, administrators risk some data loss. The-re aThe-re thThe-ree scenarios for the maintenance case. Firstly: The host is only briefly switched off or restarted – in this case the "Ensure accessibility" variant should be used. This will ensure that the VMs, which access data in the VSAN datastore or run on the host,
will remain available during the reboot. A partial data migration is performed auto-matically and this is the fastest method of ensuring that this happens. However, it is recommended that this should really only be considered as a temporary action, be-cause while the availability of the VMs is indeed ensured, there is no longer any full protection against hardware failure. If a host is to be removed from the cluster, then the "Full Data Migration" variant should be used. This may take some time, depending on how many VMs are operated on the host with a VSAN datastore. The third variant is the "No Data Migration": This means that a reboot or shutdown of a vSphere host is carried out without further demand and wi-thout VM migration, and thus cannot result in unavailability or data loss while machines are running. This method is not recommen-ded by VMware though!
The licensing for VMware's VSAN is based on the scheme already being used by vS-phere: Sockets – i.e. without limitation of cores. An example of this is: 1 server host with 2 CPUs with 8 cores each therefore needs 2 vSphere (regardless of the edition) or VSOM or vCloud Suite licenses, as well as 2 VSAN licenses. As mentioned previously, for high-availability configurations a mini-mum of 3 nodes would be needed due to requirement for a ‘witness node’. DataCore SANsymphony-V offers a simpler per server model and only requires two nodes for high-availability. In addition, the licensing is based on systems (i.e. per server node) and not on the number of processors in the system.
forms a component of the Hypervisor. The argument for this approach runs as follows: the more closely this function is linked to the Hypervisor, the higher the system through-put. But in the first version at least, VSAN does not exploit this very well. VMware quotes performance figures of only 2 million IOPS in a network with 32 nodes. Another disadvantage of this methodology is seen in the portability aspect. If the functionality is located in the Hypervisor, then it will only work for this Hypervisor – and possibly even only for a specific version.
Nutanix
Nutanix has chosen another approach for its storage. The Nutanix virtual computing platform is a converged infrastructure so-lution that consolidates the compute
(ser-ver) tier and the storage tier into a single, integrated 2U hardware-based appliance. While it positions itself as software-defined storage, in reality the entire appliance has to be purchased and deployed as a whole and while modular, there is a considerable difference in flexibility as compared to pure software based approaches like VMware or DataCore that allow hardware interchange-ability.
With Nutanix, the storage functionality is provided in a converged virtual system which is pre-packaged and, which works with the applications and the associated guest operating systems on the same level as the "normal" VMs, so to speak. The compa-ny can therefore also adapt such a VM more easily for other hypervisors – since Nutanix supports not only vSphere environments, but also, for example, configurations based
NT4Admins – the German source for technical content
www.nt4admins.de addresses professio-nals (system operators, administrators, help desk staff) and provides independent, tech-nical oriented information about the use of Windows-based platforms. Subscribers even receive scripts developed and provided by the nt4admin team in order to facilitate administrator’s life.
Rainer Huttenloher has been active in IT journalism for over 30 years. The graduate Electrical Engineer worked for major
Ger-man IT publications like Markt & Technik, Design & Elektronik, Unix Magazin, Elekt-ronik, Systeme, etc. and served as editor in chief for NT Administrator, LANline and the IT weekly Computerzeitung. Since 2009, Rainer Huttenloher works independent and cooperated with the online publica-tions speicherguide.de and virtualisierungs-guide.de before he developed and realized www.nt4admins.de in 2009 where he takes the role of a publisher and editor in chief.
on Hyper-V. However, the greatest port-ability though is offered by an approach like the one followed by DataCore where there is no dependence on the hardware nor the hypervisor: the storage software runs on any standard hardware (x86-based servers) and supports any storage and there is no limitation to use a specific Hypervisor en-vironments.
For more information, please visit the following sites:
• VMware VSAN:
http://www.vmware.com/products/ virtual-san
• Nutanix Converged Platform: http://www.nutanix.com/ • DataCore Virtual SAN:
http://www.data core.com/products/ features/virtual-san
Picture 5. DataCore supports high availability via metro-wide mirroring and remote replication for even longer distance disaster recovery. Source: DataCore
