Step by Step Guide To vstorage Backup Server (Proxy) Sizing

(1)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Tivoli Storage Manager for Virtual Environments V6.3

Step by Step Guide To vStorage Backup Server (Proxy) Sizing

12 September 2012

1.1

(2)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Revision History

Revision Number

Revision Date

Summary of Changes Changes

marked 1.0 08/22/12 Final version

1.1 09/12/12 Formatting changes and typos Added “constraint checking” section

Disclaimer

The information contained in this document is distributed on an "as is" basis without any warranty either expressed or implied.

This document has been made available as part of IBM developerWorks WIKI, and is hereby governed by the terms of use of the WIKI as defined at the following location:

http://www.ibm.com/developerworks/tivoli/community/disclaimer.html

(3)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

1. Introduction

Tivoli Storage Manager for Virtual Environments (TSM-VE) is a feature of the Tivoli Storage Manager product family for backing up virtual machines in a vSphere (VMware) environment. Tivoli Storage Manager for Virtual Environments uses the latest backup technology provided by VMware, called VStorage API (also known as “VADP” or “VStorage APIs for Data Protection”).

An essential component of Tivoli Storage Manager for Virtual Environments is the VStorage Backup Server which performs the data transfer from the ESX datastores that contain the virtual machine data to the Tivoli Storage Manager server. The VStorage Backup Server offloads the backup workload from the ESX server and acts as a proxy for a backup. Throughout this document, the VStorage Backup Server will be referred to as the "proxy ". A proxy that is configured on a virtual machine is referred to as a “virtual proxy”, and if configured on a physical machine is referred to as a “physical proxy”.

When you consider a backup solution using Tivoli Storage Manager for Virtual Environments, one of the frequently asked questions is how to estimate the number of proxies required for a specific environment. This paper guides you through the estimation process.

(5)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

1.1 Overview

The proxy estimation method described in this document is intended to help you plan a deployment of Tivoli Storage Manager for Virtual Environments. A recommended approach is described. However, there are many variations depending upon customer preferences, infrastructure capabilities, and other factors. Different vendors use various approaches to determine the number of proxies required, and may be constrained by product design. Tivoli Storage Manager provides flexibility for deploying the proxies and selecting virtual, physical, or a combination of both proxies. The intent is to provide a starting point for initial estimation and solution architecture.

The proxy estimation process comprises the following steps:

 Define how the backups are scheduled.

 Decide whether to use virtual machine proxies, physical proxies, or a combination of both.

 Estimate the number of proxies required.

 Check for any constraints in the environment based on the assumptions used in the estimate.

1.1.1 Performance

Estimating the number of proxies requires some assumptions about the performance characteristics of individual backup processes. Tivoli Storage Manager for Virtual Environments uses efficient disk block-level I/O for the backup process, and the backup process itself consumes minimal CPU and memory resources. Backup performance is determined primarily by the following system characteristics:

• I/O capabilities of the datastore storage arrays

• Back-end storage device used by the Tivoli Storage Manager server, for example, Virtual Tape Library (VTL) or disk

• Infrastructure connectivity, for example, Storage Area Network (SAN) or Local Area Network (LAN) bandwidth

It is recommended that you use benchmarking to refine the estimate of backup throughput specific to your environment.

The throughput capabilities can range significantly depending upon the environment. Observed throughputs have ranged from 40GB/Hour to well over 200GB/hour for a single, individual backup processes.

1.1.1.1 Deduplication

Tivoli Storage Manager client side (inline) deduplication is highly effective with Tivoli Storage Manager for Virtual Environments and can substantially reduce back-end storage requirements as well as the proxy to Tivoli Storage Manager server bandwidth requirements. Client side deduplication requires additional

processing (by the proxy host) that will slow the backup throughput. For a specific amount of data to backup, you may require more proxies to meet a given backup window when using deduplication as compared with not using deduplication. Generally the benefits of storage and bandwidth reduction will outweigh the cost of additional instances of proxies. For estimation purposes, you can assume that backup throughput when you use client deduplication is approximately 50% of the throughput without deduplication.

(6)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

1.1.1.2 Data transfer/transport methods

The methods used for data transfer from datastore to proxy and from proxy to the Tivoli Storage Manager server can have an impact on the per-process performance of Tivoli Storage Manager for Virtual

Environments backups and restores.

The following diagram shows the data transfer/transport methods:

The following information on the methods available is listed here for reference. TSM user documentation should be referenced for more details on the methods available and how to configure.

Data I/O from ESX datastore to Proxy Transport Method Available to Virtual

Proxy?

Available to Physical Proxy?

Comments

NBD Yes Yes Uses LAN connection

NBDSSL Yes Yes Uses LAN connection

SAN No Yes Uses direct SAN connection to

datastore (for SAN-attached datastores only).

HOTADD Yes No Uses SAN connection (via ESX host)

(7)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Data I/O from Proxy to Tivoli Storage Manager server Communication Method Available to Virtual Proxy? Available to Physical Proxy? Comments

LAN Yes Yes Data transfers over LAN to Tivoli

Storage Manager server

LAN-free No Yes Data transfers over SAN to Tivoli

Storage Manager server storage pool devices (Tape or Virtual Tape) Note: LAN-free with disk is possible using SANergy or GPFS.

1.1.1.3 Estimated ranges for performance

The throughput capabilities can range significantly depending on the environment, transport/data transfer methods, and whether client side deduplication is used. For the purpose of providing a conservative estimate, the following values are used in this document:

• 100GB/hour without client-side deduplication

• 50GB/Hour with client-side deduplication

1.1.2 Periodic Full Backup

As a best practice for Tivoli Storage Manager for Virtual Environments V6.3, you should perform a periodic full backup every 7-14 days, and regular daily incremental backups. (Incremental backups use VMware’s Change Block Tracking feature.) Full backups can be scheduled less frequently, however restore performance may increase with less frequent full backups.

1.2 A Few Definitions

Term Definition

Proxy The host that performs the offloaded backup. This host can be a virtual or physical machine. Also called “VStor Backup Server” (VBS) or “Backup Server” (BUS). The Tivoli Storage Manager Backup/Archive Client is installed on this host and provides the VMware backup function.

Datamover An individual backup process that performs the VMware guest backups. Each datamover is associated with one or more Tivoli Storage Manager backup schedules. Typically there will be multiple datamovers per proxy to fully utilize the proxy host resources. Also called “backup process”.

1.3 Scope of this document

(8)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

help to provide a successful deployment by establishing a quantitative basis for determining the quantity, placement, and sizing of the proxy hosts. There are many assumptions made within this document and actual results can vary significantly depending upon the environment and infrastructure characteristics. Careful evaluation of the environment is necessary and benchmarking during the planning phase is strongly encouraged to characterize the capabilities of the environment.

1.3.1 External Dependencies and Assumptions

The estimation process in this document is based on the assumption that no constraints exist in the

environment, and that storage capacity per VM, ESX host, and cluster are consistent across the environment. If there are large disparities in storage capacity for individual VM’s, for example, a separate sizing may need to be done for the largest VM’s. The main assumptions are:

1. datastore I/O capacity is sufficient to sustain the backup workloads. 2. LAN or SAN connectivity is sufficient to sustain the backup workloads.

3. Tivoli Storage Manager server and storage device capacity and the number of instances are designed and configured to sustain the backup workload provided by the proxies.

After completing an initial sizing estimate, it is important to validate the assumptions and other constraints that may exist in the environment. For example, more than one Tivoli Storage Manager server may be required to manage the workload to achieve the overall throughput requirement. Tivoli Storage Manager server sizing is beyond the scope of this document.

1.3.2 Performance optimization and bottleneck analysis

This document does not address design for performance or bottleneck analysis techniques for a Tivoli Storage Manager for Virtual Environments environment.

1.3.3 Proxy Hardware Configuration

Although some guidelines are provided for proxy host resource requirements, it is not the intent of this document to provide specific guidance on hardware or system configurations of physical or virtual proxy hosts. Hardware configuration (or in the case of a virtual machine, resource allocation) should be defined by a qualified system engineer that is familiar with hardware capabilities, I/O throughput, and other system requirements.

IBM Techline provides a service for pre-sales configuration of Tivoli Storage Manager hardware including Tivoli Storage Manager for Virtual Environments proxy sizing. Consult your IBM Tivoli sales representative for more information.

1.4 Scheduling of Backups

(9)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

As with any backup technique, backup results should be monitored regularly to ensure that all VM’s are backed up according to business requirements.

1.4.1 “Rotating Fulls” backups by ESX Host

All the VM’s on each ESX host are backed up once a week (full), and an incremental backup is scheduled daily for six days. For more information on how to set up the schedules, see

https://www.ibm.com/developerworks/wikis/display/tivolistoragemanager/Recommendations+for+Scheduling+ with+TSM+for+Virtual+Environments.

When you schedule backups on a per-ESX host basis, the VMs on each ESX host are backed up serially. A proxy may not be able to back itself up, so if you use a VM proxy, exclude the VM proxy from its own backup schedules.

1.4.2 Alternate Scheduling Methods

You can use other backup scheduling methods, based on business requirements or other characteristics of the environment.

1.4.2.1 “Batched Fulls”

Full backups for all ESX host/VM’s are scheduled during an extended backup window (e.g., on the weekend). The remainder of the week, all VM’s are backed up using incremental. This is usually suitable for smaller environments in which a full backup for all VM’s can be completed within a 20-40 hour window.

Typically, the full batch backup window requires a larger number of proxies than the incremental backups, so the proxies are sized for the full backup window. The estimation technique and calculation method is the same as described for “Rotating Fulls” except that the backup workload consists of 100% of all the VM’s data.

1.4.2.2 “Rotating Fulls” by VM

This method will require backup scheduling by individual VM and can include concurrent backup of multiple VM’s on the same ESX host. Although it is possible to do this with the TSM scheduler, it is usually not practical for large environments. Custom scripting methods can be used, via VMware’s PowerCLI to obtain VM lists and drive TSM backup commands (“dsmc backup vm”) via a command line.

IBM Tivoli Lab Services provides an offering for custom backup scheduling based on specific customer criteria (such as balancing load within clusters, between datastores, etc.).

(10)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

2. Step by Step Proxy Sizing

This section provides the steps for sizing a proxy for a specific deployment scenario across an entire data center. The same approach may be applied individually to separate environments that may have different characteristics, such as the average VM size.

2.1 Assumptions

 Reasonably equal distribution (within 20%) of utilized virtual machine storage capacity (datastores) across all ESX hosts.

 Backups are scheduled on a per ESX host basis. See scheduling section for more information.

 Full backup is scheduled weekly and an incremental backup is scheduled 6 days a week. This means that on any day, 1/7th of the ESX hosts will have a full backup, and 6/7th of the ESX hosts will have an incremental backup. Since we assume an even distribution of storage and VMs across all ESX hosts, this means that 1/7th of the total amount of data is backed up daily (via the full backups) and the remainder of the data is backed up incrementally.

2.2 Example environment

The following example environment is used to illustrate the estimation process:

Environment Description

Total Number of virtual machines 5000

Average Utilized Storage per VM 50GB

Total Utilized Storage 5000 * 50GB 250,000 GB

Number of ESX Hosts 250

Number of DRS Clusters 50

Backup Window 10 Hours

Assumed daily change rate 2%

2.3 Perform the Estimate

The following sections demonstrate the estimation process.

2.3.1 Determine daily backup workload

(11)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Determine Daily Backup Workload

Daily backup workload from full backups

250,000GB ÷ 7

NOTE: periodic full every 7 days

35,700GB/Day

Daily backup workload from incremental backups

(6 ÷ 7) * 250,000 * 0.02 NOTE: 6/7th of total data multiplied by incremental change rate of 2%

4,300GB/Day

Total Daily Backup Workload 35,700 + 4,400 GB 40,100GB/Day

2.3.2 Calculate Aggregate Throughput Requirement

The aggregate throughput requirement is determined by the total amount of data that is backed up daily that was computed in the previous section. We assume a 10 hour daily backup window. The goal is to meet the overall backup window, and we can adjust the number of datamovers appropriately to achieve this.

Calculate aggregate throughput requirement

Total daily backup workload 35,700 + 4,400 GB 40,100GB

Backup window 10 hours

Aggregate throughput required 40,100GB ÷ 10 hours 4010 GB/Hour

2.3.3 Calculate the number of concurrent datamovers (backup

processes)

The number of datamovers required is simply the aggregate throughput requirement divided by the estimated per-process (per datamover) throughput. Assume the per-process throughput is 100GB/hour.

Calculate number of concurrent datamovers

Aggregate throughput required 40,100GB ÷ 10 hours 4010 GB/Hour

Per process throughput estimate

100GB/Hour Number of datamovers (backup

processes)

(12)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

2.3.4 Determine the number of proxy hosts required

To determine the number of proxy hosts required, we need to know how many concurrent datamovers (backup processes) can run on a single proxy host. For estimation purposes, you should plan for no more than ten concurrent datamovers. (The maximum number of concurrent datamover processes is dependent upon proxy host resources. See the section on proxy resource requirements. However, an excessive number of concurrent datamover processes will cause performance to degrade.) We will discuss resource requirements later in this document, and you may decide to run more or less concurrent processes per proxy, but this will provide us a reasonable estimate to start with.

Determine the number of proxy hosts required

Number of datamovers (backup processes)

4010GB/Hour ÷ 100GB/Hour 40

Number of concurrent datamovers per proxy

10

Number or proxy hosts required 40 ÷ 10 4

2.3.5 Summary

We have finished crunching through the numbers to estimate the number of proxies required for a TSM-VE deployment. This gives us a good starting point, but now we need to think more about the architecture of the overall solution to determine if any adjustments are necessary. We will cover this in the next section. Here is a table that summarizes all of the steps up to this point:

Environment Description

Total Number of virtual machines 5000

Average Utilized Storage per VM 50GB

Total Utilized Storage 5000 * 50GB 250,000 GB

Number of ESX Hosts 250

Number of DRS Clusters 50

Backup Window 10 Hours

Assumed daily change rate 2%

Determine Daily Backup Workload

Daily Backup Workload from Full

Backups

250,000GB ÷ 7

NOTE: periodic full every 7 days 35,700GB/Day

Daily Backup Workload from Incremental Backups

(6 ÷ 7) * 250,000 * 0.02 NOTE: 6/7th of total data

multiplied by incremental change rate of 2%

4,300GB/Day

Total Daily Backup Workload 35,700 + 4,400 GB 40,100GB/Day

(13)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Total Daily Backup Workload 35,700 + 4,400 GB 40,100GB/Day

Backup Window 10 hours

Aggregate throughput Required 40,100GB ÷ 10 hours 4010 GB/Hour

Calculate Number of Concurrent Datamovers

Aggregate throughput Required 40,100GB ÷ 10 hours 4010 GB/Hour

Per Process Throughput Estimate 100GB/Hour

Determine Number of Proxy Hosts Required

Number of datamovers (backup

processes)

Number of concurrent datamovers per proxy

10

Number or proxy hosts required 40 ÷ 10 4

2.4 Constraint Checking and Architectural Considerations

Now that we have an estimate of the number of proxies required to achieve the daily backup workload, we must now consider whether this makes sense in practical terms. TSM-VE provides a great deal of flexibility in deployment options, so we need to determine which options makes the most sense. We will consider other factors and determine if adjustments are required.

2.4.1 Check for Constraints

(14)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Constraint Validation

Proxy i/o throughput: Can each individual proxy sustain the required i/o throughput?

Ensure that the proxy can be configured with sufficient adapter cards (NICs and HBAs) to support the required throughput.

TSM server throughput: Can the TSM server support the required aggregate i/o throughput for all of the proxies?

Ensure that the TSM server is configured to support the required aggregate throughput. Multiple TSM servers may be required in some cases.

TSM server sessions: Can the TSM server support the required number of concurrent backup sessions from all of the datamover processes?

Ensure that the TSM server is configured to support the required number of concurrent backup sessions.

Infranstructure bandwidth: Can the LAN or SAN accommodate the aggregate workload required for all of the backup processes?

Ensure that the LAN and SAN networks have sufficient bandwidth to accommodate the backup (and restore) bandwidth requirements.

Datastore I/O Capacity: Can the ESX datastore accommodate the i/o required to support the required backup throughput?

Ensure that the Datastore I/O devices are capable of supporting the I/O data transfer rates required for the backup processes. The assumption for the per-process backup throughput may need to be adjusted. Per-ESX host backup window: Will an ESX host

have an excessive number of VM’s or storage capacity that cannot be backed up within the required window? This constraint applies when schedules are created based on ESX hosts. Since VM’s are backed up serially for each ESX host, it is possible to exceed the backup window if the total storage capacity for an ESX host significantly exceeds the average.

Determine if it is possible for any one ESX host to have an excessive number of VM’s and storage that will result in not achieving the backup window.

2.4.2 Additional capacity requirements

We have estimated the number of proxies based only on the backup requirement during the backup window (in our case 10 hours). However, full image restores of virtual machines require the use of a proxy as well. The proxy can be located anywhere within the datacenter and could be a separate, dedicated proxy used only for restore purposes.

If you intend to use the same proxies for image restores during the backup window, you will need to add this to your workload estimate which may increase the number of proxies required. You may also want to consider a spare proxy in the event of a failed proxy host.

2.4.3 Physical or virtual proxy?

The first consideration is to determine whether physical or virtual proxies will be used. Note that you don’t need to decide all one or the other. It is entirely possible to use virtual proxies for part of an environment, and physical proxies for another where it makes sense.

(15)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

2.4.3.1 Questions to ask when you decide between a physical and a virtual proxy

Following is a list of questions you should consider when deciding between physical and virtual machines showing which type of proxy would be preferred. If the answer to the question is “Yes” then preference should be given to the type of proxy in the “Yes” column. . If the answer to the question is “No” then preference should be given to the type of proxy indicated in the “No” column.

Question Yes No

Do you require backup traffic to flow over the SAN as much as possible?

*Note: Virtual machine proxies can take advantage of Hotadd data transfers from a SAN datastore to the proxy which primarily uses SAN I/O via the ESX host HBA. However, a virtual machine proxy cannot take advantage of LAN-free data transfers from the proxy to the Tivoli Storage Manager server.

Physical* Virtual

Does your LAN (IP Network) have sufficient bandwidth to accommodate the backup traffic.

Virtual Physical

Do you want to use LAN-free data transfers from the proxy to the Tivoli Storage Manager server?

Note: LAN-free is usually only used with Tape or Virtual Tape backup storage devices.

Physical Virtual

Do you prefer or require that all new hosts are virtual and not physical machines? Virtual Either Do you want to minimize the number of proxy hosts?

Note: The preference is based on the assumption that you will dedicate more resources to a physical proxy than a virtual proxy.

Physical Virtual

Do you use NFS attached datastores? Virtual Either

Is 10Gbit Ethernet connectivity available to the Tivoli Storage Manager server? Virtual Either

2.4.3.2 ESX Clusters and distribution of virtual proxies

It is important to consider the distribution of virtual proxy hosts within the infrastructure. In the example, there are 4 proxies that backup 50 ESX hosts each, or 10 ESX clusters. Although you can achieve the desired throughput with 4 proxies, you may want to consider using at least one virtual proxy per ESX cluster. This is because a virtual proxy within an ESX cluster can access datastore storage via Hotadd. Hotadd provides an efficient (and low overhead) data transfer method. For SAN attached storage, the proxy transfers data directly through the ESX host’s fibrechannel adapter.

Using the example, you can use 10 proxies to cover all 10 ESX clusters. You can reduce the number of data movers per proxy to distribute the workload over a greater number of proxies and reduce the resource requirements for each proxy. Using 10 virtual proxies will also provide additional “reserve” capacity for restores which occur during the backup window.

2.4.3.3 Deduplication choices: TSM Deduplication vs. Appliance Deduplication

(16)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

3. Your Estimate

You can use this table to provide your proxy sizing estimate, using the example as a guideline:

Environment Description

Total Number of virtual machines ___________

Average Utilized Storage per VM ________GB

Total Utilized Storage _____ * ___ GB

# of VMs * Avg. Storage Per VM _______ GB

Number of ESX Hosts _____

Number of DRS Clusters ___

Backup Window ___ Hours

Assumed daily change rate __%

Determine Daily Backup Workload

Daily Backup Workload from Full

Backups

______GB ÷ 7

NOTE: periodic full every 7 days _______GB

Daily Backup Workload from Incremental Backups

(6 ÷ 7) * ______ * 0.0_ NOTE: 6/7th of total data

multiplied by incremental change rate of __%

_________GB

Total Daily Backup Workload _______ + ______ GB ______GB

Calculate Aggregate Throughput Requirement

Total Daily Backup Workload (from previous calculation) ______ GB

Backup Window ______ hours

Aggregate throughput Required ______ GB ÷ ___ hours ______ GB/Hour

Calculate Number of Concurrent Datamovers

Aggregate throughput Required ______ GB ÷ ___ hours ______ GB/Hour

Per Process Throughput Estimate 100GB/Hour

______ GB/Hour ÷ 100GB/Hour ______

Determine Number of Proxy Hosts Required

Number of datamovers (backup

processes)

______ GB/Hour ÷ 100GB/Hour ______

Number of concurrent datamovers per proxy

10

(17)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

4. Proxy Host Resource Requirements

Resource requirements for a proxy are driven by the following key factors:

 I/O data transfer capacity

 CPU capacity

Of the two factors, I/O capacity is the most important factor because the proxy’s main role is to move data. When you use client deduplication, the CPU resources may become the constraint for throughput.

4.1 Determining proxy resource requirements

4.1.1 Determining I/O resource requirements

In the case of LAN data transfers the critical I/O resources for the proxy are the network adapter cards (NICs). In the case of SAN data transfers (which includes SAN transport from the datastore to the proxy and LAN-free from the proxy to the TSM server), the FC adapters (HBA’s) are the critical I/O resource. When both LAN and SAN transfers are used, then both NICs and HBAs are critical to support the required data transfer rates. An example of this is when LAN transport is used between the datastore and proxy, and LAN-free is used between the proxy and the TSM server.

For virtual proxies, the resource requirements apply to virtual adapters and the requirements (number and speed of adapters) will remain the same as a physical proxy. However, dedicating shared resources across an ESX hypervisor may require additional planning and configuration for a virtual machine to ensure that sufficient resources are available to other VM’s hosted by the same ESX host.

The NIC and HBA adapters should be sized to ensure adequate capacity to handle the expected i/o data rates through the IP network and SAN, respectively.

Machine backplane i/o capacity must also be considered, but generally is not an issue for a properly configured system.

4.1.2 Determining CPU requirements

For estimation purposes, plan for 50% of a 2.2Ghz physical or virtual processor cores (or equivalent) per individual backup process when client-deduplication is not used. Thus, a proxy that runs 8 concurrent backup processes would require 4 processor cores.

Note: Processor core refers to a processing unit. A quad-core socket has 4 processor cores.

When you plan for deduplication, 150% of a 2.2Ghz processor core should be configured per backup process. For example, a 16 core proxy host should have no more than 10 concurrent backup processes. An advantage of considering virtual proxies is that you can distribute a larger number of proxies with smaller resources to accommodate the same backup workload as fewer, physical proxies,that require more

resources each.

4.1.3 Memory estimation

(18)

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

(for example, Windows 2008R2). A minimum of 4GB of RAM should be considered when running four concurrent backup processes, with an additional 1 GB for each additional backup process.

Step by Step Guide To vstorage Backup Server (Proxy) Sizing

Tivoli Storage Manager for Virtual Environments Guide to Proxy Sizing

Tivoli Storage Manager for Virtual Environments V6.3