Technical white paper
Disaster recovery scenario guide for using HP StoreOnce Backup systems and Symantec
NetBackup 7.x
“Peace of mind when you need it most – the power of HP StoreOnce Catalyst when Disaster
Strikes!”
Contents
Executive Summary
2
Introducing HP StoreOnce Emulations and Symantec NetBackup 7.x integration
3
Introduction to Symantec NetBackup and OST
4
NetBackup Terminology 4
Introduction to OST/StoreOnce Catalyst 5
Licensing considerations
7
Symantec licensing 7
Licensing modes 7
VTL licensing 8
How this relates to our integration example 8
HP StoreOnce B6200 licensing 8
Component 9
NetBackup License required 9
Source B6200 – License required 9
Target B6200- License required 9
Comment 9
Scenario 1 –Total loss of Primary site
10
Preparing the DR environment 11
StoreOnceVTL policy restores 13
Setting up the StoreOnce environment 13
Making the Master Server aware of the replicated libraries 14
Restoring the VTL libraries to the DR Media Server, Valiant 22
An Alternative Approach 24
StoreOnce NAS policy restores 25
StoreOnce and AD environment 25
Making the Master Server aware of the replicated shares 29
Recovering the image to the Disaster Recovery server, Valiant 33
StoreOnceCatalyst policy restores 34
Scenario 1 conclusions 38
Scenario 2 – Medium term loss of backup facilities at the Primary Site
39
Creating the Storage Unit Group 40
Creating the Storage Life Policy 42
Scenario 3: Restoration of a NEW Primary site from the Secondary site
45
Executive Summary
There is no point in testing your data recovery strategy when you are in the middle of a disaster! You want to know it is proven in advance and how to design the environment to provide a quick recovery Service Level. This whitepaper provides real world testing to give you confidence when using the HP StoreOnce Backup system and Symantec NetBackup 7.x Disaster Recovery in three specific Disaster Recovery scenarios.
The first illustrates a Disaster Recovery scenario that restores all possible StoreOnce devices: VTL emulations, NAS CIFS share emulations and StoreOnce Catalyst stores. Although all three work effectively in this scenario, a significant learning is that the StoreOnce Catalyst Disaster Recovery process is the simplest, most coherent and easiest to use. The second two scenarios provide further examples of the StoreOnce Catalyst Disaster Recovery process.
The following summary highlights the key learnings:
HP StoreOnce VTL emulations used in Disaster Recovery scenarios work as expected but require additional steps to complete recovery. It is necessary to present devices to the Media Server, perform an inventory and “import” replicated cartridges into the NetBackup catalog before recovery can take place. See Scenario 1, StoreOnceVTL policy restores.
HP StoreOnce CIF share emulations used in Disaster Recovery scenarios work as expected but require additional steps to complete recovery. It is necessary to create user access permissions and to “import” the share contents into the NetBackup catalog before recovery can take place. See Scenario 1, StoreOnce NAS policy restores.
The use of HP StoreOnce Catalyst technology and OST integration in NetBackup provides the fastest method of Disaster Recovery because the NetBackup catalog is aware of all copies of the backup data that exist. See Scenario 1, StoreOnceCatalyst policy restores. Using the Low Bandwidth backup mode of HP StoreOnce Catalyst and OST has additional benefits. By making use of Symantec NetBackup Storage Unit Groups, it provides an automated, fail-over solution should the primary site suffer an extended outage of the backup device. See Scenario 2, Scenario 2 – Medium term loss of backup facilities at the Primary Site
Providing the Disaster Recovery process and Media Server connectivity is designed in from the start, “site role reversal” is very easy to implement using OST/HP StoreOnce Catalyst technology. See Scenario 3, Scenario 3: Restoration of a NEW Primary site from the Secondary site
HP StoreOnce catalyst can also be used to meet demanding data offsite SLA’s by replicating data offsite first then replicating back to the Primary site for local recovery – this is also explained in Scenario 2 – Medium term loss of backup facilities at the Primary Site .
NetBackup Licenses (Enterprise Disk) and HP StoreOnce (catalyst store licence) are required to implement the OST/HP StoreOnce Catalyst DR solutions but these are well worth the investment for the peace of mind and flexibility these solutions bring. The solutions can also lead to cost savings in other areas by re-purposing backup storage rather than over-provisioning too much backup storage.
This Disaster Recovery Scenario guide is a follow on from a previous HP StoreOnce/NetBackup integration guide, HP StoreOnce VTL, CIFS share and OST/Catalyst integration guide with Symantec NetBackup 7.5, EJ022-90953 , which describes how to configure HP StoreOnce VTL, NAS(CIFS) and Catalyst Stores and associated. It is strongly recommended to read the HP StoreOnce/NetBackup integration guide prior to
reading this disaster scenario guide.
Introducing HP StoreOnce Emulations and Symantec NetBackup 7.x
integration
HP StoreOnce Backup systems belong to a range of disk-based backup systems supporting in-line deduplication, source-side deduplication and deduplication-enabled replication that offer users the following business benefits:
Data growth is managed more effectively by implementing deduplication technology (developed by HP Labs) to store more backups on the same physical hardware for longer periods of time.
Implementing a disaster recovery solution is more cost effective.An additional benefit of deduplication is that it enables low bandwidth replication. Data can be replicated offsite over low bandwidth (cost-effective) links - making disaster recovery more affordable than ever before.
Three types of emulation are available – VTL, NAS shares and StoreOnce Catalyst stores. This provides the widest range of functionality and capabilities in a single appliance. See Table 1 below.
Comparison table of emulations; showing key features and when to use
Emulation Key Features Best used in Comments
Virtual Tape
Uses virtual tape drives and virtual slots to emulate
physical tape libraries
Enterprise FC SAN environment (B6200 and 4xxx series).
HP StoreOnce also supports iSCSI VTL (4xxx and 2xxx series)
Tried and tested, well understood but lacks sophistication. Uses NetBackup Robot and Drives
device type
NAS (CIFS/NFS shares)
NAS shares can be easily configured and viewed by the
operating system
Specific environments that do not support tape emulation backup or prefer to backup directly to disk.
In some cases the licensing may be lower cost for NAS shares as a
backup target.
This is a NAS target for backup - not recommended for random NAS file
type access.
Uses NetBackup Basic Disk device type
OST with StoreOnce Catalyst
NetBackup has total control over the HP StoreOnce appliance, providing source
based deduplication, replication control, improved
DR etc.
Environments that require a single management console for
all backup and replication activities and the ability to implement federated
deduplication*
Requires HP OST 2.0 plugins to be installed on NetBackup Media servers
Uses NetBackup OpenStorage device type
Table 1: HP StoreOnce Emulations
* Federated deduplication is an HP term referring to the ability to distribute the deduplication load across NetBackup Media Servers. This feature is sometimes known as source-based deduplication or low bandwidth backup.
Introduction to Symantec NetBackup and OST
Symantec NetBackup is an Enterprise Class data protection application. Its architecture is designed for a large and complex distributed computing environment. NetBackup provides scalable storage servers that can be configured for a variety of tasks such as backup, recovery, archiving and file migration. NetBackup provides a variety of client agents for different operating systems and applications. There is a variety of additional functions, such as Advanced Disk and the implementation of the Symantec Open Storage Technology (OST) Application Programming Interface (API). NetBackup Media Servers can use disk or tape for storage targets.
NetBackup Terminology
NetBackup Master Server: Manages backups, archives and restores. A Master Server manages all policies for a NetBackup domain and is required for all backup and restore operations. It is responsible for all media (can be tape or disk) selection and maintains an internal database called a Catalog. The Catalog tracks backups and media and is used to quickly locate the correct media and backup items. There is one Master Server per NetBackup domain, but they can be clustered for redundancy.
NetBackup Media Server: Distributes the backup load across large deployment configurations. Storage devices are attached to Media Servers via SCSI, SAN or network connection. It is possible for a Media Server to be present on the same physical server as the Master Server. A Master Server can control many Media Servers and also function as a Media Server. It is possible in large enterprise scale operations to have multiple Master Servers.
Storage Server: When disks are connected directly to the Media Server I/O. In this configuration, disks can be configured in an OpenStorage Disk Pool.
Storage Units: The logical unit that is used as the target device for policies and Storage Lifecycle Policies. Disk Pool: A collection of disk volumes presented to multiple Media Servers.
Clients: Servers or workstations that have the NetBackup client software loaded and will back up over the network to their designated Media Server.
Policy: Backup job
Storage Lifecycle Policy (SLP): This is a policy that can be overlaid onto a standard policy. It controls creation, and duplication (multiple copies) of backup data and maintains where backups are stored and their retention period.
Figure 1 shows how the NetBackup components are interconnected.
HP StoreOnce emulations roughly translate into the following Symantec device terminology: StoreOnce VTL = NetBackup Robots and Drives Device
StoreOnce NAS CIFS shares = Basic Disk. Locally attached disk storage and network attached disk storage are presented as file systems to the Media Server. The directory structure is specified when the storage unit is created. This is the standard configuration for StoreOnce NAS. Basic Disk cannot be used in a Storage Lifecycle Policy.
StoreOnce Catalyst Stores = OpenStorage (HP StoreOnce Catalyst). This is used for intelligent storage devices that are OST protocol aware. This option is vendor specific and requires the creation of a Symantec Storage server and disk pool.
Introduction to OST/StoreOnce Catalyst
OST stands for Open Storage Technology – it is a Symantec specific interface to enable tighter integration with backup appliances of all types. The OST API allows the features of individual hardware vendors with backup appliances to be supported through a common interface within
NetBackup and Symantec Backup Exec. The appliance vendors contribute by writing what are called “OST-plug-ins”, which are installed on NetBackup Media Servers and Master Servers to complete the integration.
In HP StoreOnce terminology, a “Catalyst Store” along with the HP OST 2.0 plugin is HP’s methodology for providing a tight integration with Symantec NetBackup OST. This provides customers with the following benefits:
Backup and restores are performed to specific NetBackup Open Storage devices providing improved reporting capabilities (deduplication ratios, etcetera).
Part of the deduplication process can be offloaded onto Media Servers, so less network bandwidth is consumed within a Data Center or over a Wide Area Network (WAN) link when backup data is sent to an HP StoreOnce B6200 appliance. This is only achievable when using the OST API with Catalyst API integration. Offloading the deduplication workload to a NetBackup Media Servers will improve the overall throughput that the B6200 StoreOnce Backup system can deliver.
Replication of OpenStorage devices (in NetBackup terminology this is called Duplication) is controlled by NetBackup – rather than the appliance itself – leading to the backup Catalog having true visibility of all copies of the backup data that exist (if required). In
addition different retention periods can be specified for different copies of the data – for example, copies stored at the Data Center can have longer retention periods than those stored at remote sites.
Disaster recovery times from replicated copies are reduced because the backup Catalog has visibility to all copies of the data.
All in all, OST with Catalyst provides the customer with a comprehensive and intelligent backup solution, tightly managed from the NetBackup Management Console.
The specific HP implementation is shown in Figure 2. Note the following:
The StoreOnce Catalyst plugin to the OST API provides the “bridge” between Symantec standard OST commands and Catalyst store commands.
A Catalyst store created on the StoreOnce B6200 Backup system can be configured as either “high bandwidth” or “low bandwidth”. If configured as high bandwidth, all data is passed from the NetBackup Media Server (using either a 1Gbe or 10GbE interface) to the B6200 high bandwidth Catalyst store; and all processor intensive deduplication is performed on the B6200 Backup system and not the Media Server.
If the Catalyst store on the B6200 is configured as low bandwidth, the OST/catalyst API integration ensures that the processor intensive part of deduplication (called hashing, chunking and compression) is performed on the Media Server. Low bandwidth has the advantage of transferring only a fractional amount of data over the network to the Catalyst store on the B6200 Backup system. This technique can be used to reduce network backup traffic in the Data Center or can be used to allow remote sites to backup over WAN links to a low bandwidth Catalyst store at a centralized Data Center.
Catalyst stores can be “copied” (replicated (HP), duplicated (Symantec)) with OST commands and defined NetBackup Storage Lifecycle Polices using low bandwidth techniques. This copy is done without having to “rehydrate” the data as it is sent from one location to another. Significant savings on time and bandwidth provide a very cost-effective Disaster Recovery solution for customers.
Licensing considerations
Symantec licensing
Licensing modes
Symantec operates two licensing models:
a) Standard Licensing (Traditional): This is based around the number of clients being protected. This approach is component-based on processor tiers, and separate licenses are required for each server running in the Backup Domain. Integrating a StoreOnce product into a NetBackup traditional licensing environment requires no additional NetBackup licenses.
b) Platform Base Licensing (Capacity): This is based on a concept of Front-End Terabytes (FETB). A Front-End Terabyte (FETB) is one terabyte of protected data. The data can either be on clients or devices where the software is installed or where the software is used to provide backup functionality. The total amount of protected data is calculated based on the backup image header information in the NetBackup catalog. The good news is that this capacity licensing is not related to the efficiency of the backup devices – so there is no “extra” cost for using deduplication appliances such as HP StoreOnce B6200 – unless special features like OST are involved.
In both licensing models Symantec provide a utility (nbdeployutil) that scans the environment, analyses the setup and produces either a report or an xls spreadsheet which shows the licensing configured and required.
For more information on nbdeployutil go to http://www.symantec.com/docs/TECH145972
Figure 3: NetBackup licensing overview
The table above shows the main NetBackup Licensing components based around architectures and advanced features. The Enterprise Disk license is particularly relevant in this whitepaper and is delivered in units of 1 FETB.
VTL licensing
In September 2008, NetBackup licensing of VTLs (and all disk-based backup options, such as OpenStorage and AdvancedDisk) changed. There are now two options available for licensing VTLs. Both of these are based on the amount of primary storage (Front-End Terabytes) that are backed up to the VTL. This capacity is the total sum of data, from a NetBackup perspective, included in the largest set of full backups to the VTL.
If the VTL can perform deduplication and/or replication, the Enterprise Disk Option license is required.
If the VTL cannot perform deduplication or replication, the Basic Disk license is included at no extra charge and the Shared Storage option license (per drive) is only required for sharing physical tape drives in a FC SAN.
These licenses allow a customer to configure as many virtual tape drives as desired and allows sharing of these virtual tape drives among Media Servers. Sharing of virtual tape drives is typically not necessary or wise when using a VTL. This was useful when VTLs were first introduced and some of them supported only four to six drives, but support for configuring hundreds of virtual drives has alleviated the need for sharing the drives.
NOTE: The Library Based Tape Drive Option for NetBackup is ONLY for licensing physical tape drives with NetBackup, not virtual tape drives.
How this relates to our integration example
For the integration environment in this guide let us adopt the NetBackup Platform Base capacity licensing model and assume we are protecting 16 FETB. Let us also assume that the Master Servers and Media Servers are four core processer servers.
We will show the licensing requirements for backing up the 16 FETB to different device emulations on the HP StoreOnce B6200 Backup system. Additional licenses may also be required depending on the backup technique used to backup the flat files in our Integration environment.
Simple disk agent backup VSS snapshot backup
To keep things simple we are ignoring additional licenses that would be required for special agent based backups for SQL, Oracle etc. and other special options like bare metal restore or open file backup options, We simply want to explain how to license the HP StoreOnce emulations.
HP StoreOnce B6200 licensing
The StoreOnce B6200 licensing system is fairly straightforward:
Each storage shelf arrives with a license that must be loaded, further capacity upgrades also arrive with a license included. There is no licensing required for VTL or NAS emulations
VTL and NAS replication requires a license (per couplet) on the target site, but only if VTL/NAS replication is used on that couplet OST/Catalyst devices require a license to be used at both source and target – but DO NOT require an additional replication license as well. Catalyst licensing is per couplet; if a couplet is not using Catalyst stores – you do not need the license.
A summary of licensing for the integration environment (illustrated in Figure 4) is shown in Table 2.
This example assumes the user is going to use VTL or NAS or Catalyst stores. The 16 FETB licences are common to all the emulations. Only if we use Catalyst stores do we need to purchase the Enterprise Disk license to obtain open storage support and Storage Lifecycle Policy support. In these examples we used the disk agent backup not the snapshot method of backup.
Component NetBackup License required License required Source B6200 – License required Target B6200- Comment
Master Server, Media Server and Client
1 Enterprise server
License N/A N/A
We are using four core Windows processors in our Master and Media Servers, which attracts the NetBackup Tier 3 licensing for 4-7 socketed processors.
Media Servers (2) 2 Enterprise server
Licenses N/A N/A
Any system with two or more Media Servers and all Media Server licenses have to be Enterprise Server class licenses.
VTLs 16 x1 FETB Enterprise Disk
licenses None
1 couplet replication license NAS (CIFS) 16 x1 FETB Enterprise Disk
licenses None
1 couplet replication license Catalyst Store 16 x1 FETB Enterprise
Disk licenses 1 couplet catalyst License
1 couplet catalyst License
NO replication license on target required if target already licensed for Catalyst Application server
Standard Client license if disk agent backup OR Enterprise client License if
snapshots are used
N/A N/A To avoid downtime, high-availability
systems may instead perform the backup on a snapshot—a read only copy
of the data set frozen at a point in time to allow applications to continue writing
to their data. DR Recovery server
Standard Client license if disk agent backup OR Enterprise client License if
snapshots are used
N/A N/A
Scenario 1 –Total loss of Primary site
This is a typical Disaster Recovery scenario. A test environment was set up as shown below. There are two simulated Data Centers in the same NetBackup Domain.
On the Primary Data Center site (PDC) simple Filesystem backups are performed to an HP StoreOnce Backup system (B6200A) containing three different device emulations a) VTL, b) NAS (CIFs) share, c) HP StoreOnce Catalyst store.
Replication of these three devices is configured so that the backups are replicated across a low bandwidth link to the HP StoreOnce Backup system (B6200B) at the Secondary Data Center (SDC).
A detailed description of how the devices were configured and replication was enabled can be found in the HP Whitepaper HP StoreOnce VTL, CIFs share and OST/Catalyst integration guide with Symantec NetBackup 7.5.
The main focus of this whitepaper is to explain the steps and time involved in recovering the data that was originally on the Media Server at the Primary Data Center – assuming all of the PDC has been destroyed. (This is simulated by shutting down the Media Server “heartofgold”. We assume the NetBackup Master Server is not down as a result of the loss of the PDC. The same effect would occur if the NetBackup Master Servers were in a clustered configuration.
Figure 4: Test environment for DR Scenario 1
Our challenge is to recover the heartofgold data that we backed up to B6200A, the HP StoreOnce Backup system at the Primary Data Center, to a new DR recovery server (Valiant) at the Secondary Data Center. We shall use the replicated copies of the data that exist on VTL, NAS (CIFS) and Catalyst stores on B6200B, the HP StoreOnce Backup system at the Secondary Data Center, using the Deathstar Media Server. The Heartofgold Media Server and the B6200A are no longer available because of the disaster, as shown in Figure 4.
We assume the two following steps have been implemented in advance of the DR recovery process a) Media Server “Deathstar” has been attached to Master Server “Saratoga”
b) Client “Valiant” has had NetBackup Client software installed and has had “Deathstar” added to its access list
The DR recovery process for Scenario 1 has four stages:
1. Bring the Media Server, DeathStar, and the DR server, Valiant, at the Secondary Data Center into the NetBackup 7.5 Master Server, Saratoga, environment.
2. Restore the StoreOnce VTL policy. 3. Restore the StoreOnce NAS policy. 4. Restore the StoreOnce Catalyst policy.
Preparing the DR environment
1. On the Master Server, add the Media Server “Deathstar” to the Additional servers list of the Master Server. Also, add the new Media Server to the Additional servers list of the client “Valiant”.
2. Install the NetBackup client software using the NetBackup 7.5 DVD browser Client Software installation wizard onto Valiant. The NetBackup Client installation installs the software to perform backups and restores from a user interface on the client.
3. Follow the steps in the wizard, as shown below, to “push” the client software onto Valiant from the Master Server, Saratoga.
a) b)
c) d)
4. We must also add “Valiant” to the list of Servers that can be accessed by the Master Server – otherwise, the restore will not work. The restore path will function as follows: commands from the Master Server, Saratoga, selects backup images from B6200B to be read via the Media Server, Deathstar. The Media Server, Deathstar, writes data to the DR Recovery Server, Valiant (NetBackup client).
StoreOnceVTL policy restores
The environment is now ready to start the recovery process from the StoreOnceVTL backup policy replicated copies that exist on B6200B, the HP StoreOnce Backup system, at the Secondary Data Center.
There are three stages to the recovery process.
Set up the StoreOnce environment on B6200B, so that the DR Media Server, Deathstar, can access the replicated VTL libraries across the FC SAN.
Make the Master Server, Saratoga, aware of the replicated libraries. This is necessary because the B6200A instigated the replication, not NetBackup, so the Master Server Catalog has no knowledge of them.
Recover the VTL libraries backup image on B6200B via the DR Media Server, Deathstar, to the new production DR server, Valiant.
Setting up the StoreOnce environment
The StoreOnce Replication wizard does not assign the replication target library to a FC port, so it is necessary to assign the NetBackup75vtltarget device a FC port so that it can present itself to the Media Server, Deathstar, via Fibre Channel on the Disaster Recovery site.
1. Select the NetBackup75vtltarget from the HP StoreOnce GUI and click Edit. Select Port1 from the drop down list of Ports. Click Update and the NetBackup75vtltarget is presented to port 1 as shown below.
2. Look at the Interface Information tab to find the Fibre Channel World Wide names (WWN) that have been assigned to
NetBackup75vtltarget by the HP StoreOnce B6200 Backup system. These WWNs must be entered into the zoning configuration on your
FC switch to ensure that the DR Media Server, Deathstar, can now access NetBackup75vtltarget. There is a separate WWN for the Drive and the Medium Changer.
3. Perform an operating system hardware scan on the DR Media Server, Deathstar, to check that the VTL, NetBackup75vtltarget, can be detected.
Making the Master Server aware of the replicated libraries
We must now configure NetBackup75vtltarget into NetBackup and import the replicated cartridges into the NetBackup Enterprise Media Manager database (EMM) or “Catalog” as it is widely known.
If doing an import of tape, bear in mind that the NetBackup Catalog already contains cartridge entries for both the VTL we configured and backed up to on B6200A (heartofgold TLD-01) and also the cartridges replicated to the Target device on B6200B and presented to the DR Media Server, Deathstar, as TLD1 have identical barcodes. NetBackup will not let you update the Volume configuration after an inventory on the DR Media Server, Deathstar, unless you select the cartridges in heartofgold-hcart2-robot-tld-01 and delete them. This is because NetBackup cannot accommodate multiple entries for the same barcode media and locks in cartridge barcodes with certain Media Servers. After the media associated with heartofgold (primary site) has been deleted, you can proceed to successfully update the volume configuration (as part of the device
configuration wizard described next) from the replicated copy at the DR site.
1. Start the Configure Storage devices wizard.
2. Select deathstar (the Disaster Recovery Media Server) as the host on which to configure devices.
3. Select the devices to be configured and click Next to run through the configuration wizard, as previously. Commit the configuration and create the storage unit.
4. Look under Storage Units in the NetBackup Administration Console to see the VTL library, NetBackupvtltarget, displayed as deathstar-hcart2-robot-tld-1, using the Symantec naming convention.
We now have the replicated VTL in B6200B presented to the Media Server, Deathstar, at the Secondary Data Center. An import of tapes is really only required if you replicate virtual tapes between NetBackup backup domains or if a tape has been deleted, but for the sake of completeness we will show how to perform the complete import process. See also An Alternative Approach.
6. The device NetBackup75vtltarget (identified as TLD(1) –deathstar) displays the replicated cartridges. Note how the barcodes found are identical to those on device NetBackup75 from which they were replicated.
7. Click Yes. These cartridges are now used to update the volume configuration so that the data lost from the Primary Data Center can be recovered at the Secondary Data Center to the newly-built Disaster Recovery server, Valiant.
The volume configuration has been updated but because we deleted the Media IDs on NetBackup75 to allow this inventory to take place on the Media Server, Deathstar, there are now no media ID entries in the NetBackup Catalog for the data that we backed up from heartofgold. This can be proved by doing a Catalog search (on the Master Server) where nothing is found.
8. We must now import the replicated cartridges that exist on NetBackupvtltarget using the Import Images wizard.
9. This is a two-phase process: first, we create a list of expired images and, then, we select which images we want to import in Phase II. No import occurs in Phase I.
a) Click Next to start. b) Select the Media Server, Deathstar, as the media host to perform the import operation.
c) Select Tape as the Image Type. d) In our example all the backups we require are on the tape in Slot1 which has Tape ID 093801. If we did not know, we would import ALL the media then carry out a Catalog search.
10. You can monitor Phase I of the import process by selecting the Results tab of the Catalog query page. Here we can see all the images of the heartofgold client backups from 25th June to 7th July that have been discovered on tape ID 093801.
11. Now we proceed to Phase 2 of the import process. Go to the Catalog tab and select Import as the action. Specify Deathstar as the Media Server, click Search Now and all the backups available for import are displayed – let us import only the last full backup from July 7th.
12. To import the backups that consist of fragments on multiple tapes, first run the Initiate Import (Import Phase I). The first phase reads the Catalog to determine all the tapes that contain fragments; then in Phase 2 import all the tapes identified as
13. In our case the whole backup is on a single tape so we can initiate Phase 2 (Import) directly using the import icon at the top of the Catalog toolbar. Click OK to confirm the Import.
14. Import progress can be monitored via another entry on the “Results” tab of the Catalog part of the navigation tree. It displays all the individual files associated with the backup image as it imports them. As you can see, it imported successfully.
Restoring the VTL libraries to the DR Media Server, Valiant
1. Start the NetBackup Restore Client on the Master Server, Saratoga.
2. Select File –Specify NetBackup Machines and Policy types.
3. The following pop-up appears. In this DR scenario we need to change the destination client for restores to Valiant.nearline.local . Use Edit Client list to first add Valiant to the client list, then select Valiant for the destination client for restores.
We also need to ensure the Master Server, Saratoga, is selected as the server to use for backups and restores, otherwise a pipe close failure occurs. The Master Server is controlling the restores but will use the Deathstar Media Server to access the required image (tape in this case) from the HP StoreOnce Backup system, B6200B. The source client is heartofgold (the Media Server at the Primary Data Center, where the data was originally backed up). Click OK.
4. The Restore client then displays the last backup of “heartofgold” under policy name StoreOnceVTL. Select the files you wish to restore (in this case the full 2GB dataset).
6. Check the restore to different location button and click Browse to display the Browse window where you can check that the data is being re-directed to DR server, Valiant. Click Start Restore.
7. The restore completes successfully and we have recovered the heartofgold data from the B6200B at the Secondary Data Center to a new DR Server, Valiant, at the Secondary Data Center.
In a DR scenario where VTL in B6200A (the StoreOnce Backup system at the Primary Data Center) has a failure, we must use NetBackup to manually move all the virtual tapes in VTL on B6200A out of the VTL library via the eject media command in NetBackup . Then, on the DR Media Server, Deathstar, we can initiate a scan for tapes found in VTL on B6200B (the StoreOnce Backup system at the Secondary Data Center). The virtual tapes created in VTL on B6200A should appear in VTL on B6200B and a restore can be initiated without any further action because the NetBackup Master server already knows the content of the tapes; there is no need for content import. This emulates a scenario where we move physical tapes from one physical tape library to another physical tape library be it on a different site.
Note: An import of tapes is really only required if you replicate virtual tapes between NetBackup backup domains or if a tape has been deleted and its entries in the NetBackup catlog need to be re-created.
StoreOnce NAS policy restores
We will now repeat this sequence of events for the NAS (CIFS) shares that we backed up to on B6200A and then replicated to B6200B. There are three main stages:
Set up the StoreOnce and AD environment on B6200B for the Media Server,Deathstar, so that it can access the replicated NAS shares with the correct permissions.
Make the Master Server, Saratoga, aware of the replicated shares. The B6200A instigated the replication, not NetBackup, so the Master Server Catalog has no knowledge of them.
Recover the NAS shares image on B6200B to the DR server, Valiant.
StoreOnce and AD environment
Before it can access the data that was replicated from B6200A , Deathstar must be able to connect to the NAS share, NetBackup75NAStarget, on the B6200B . It must also have access permissions to NetBackup75NAStarget configured for Active Directory. The permissions are complicated by the fact that we have two interfaces to the B6200B: a management interface on the 1GbE subnet and a data (share) interface on the 10GbE subnet. This determines from which servers the permissions can be set.
Activity Mgmt interface (1GbE) Data Interface (10GbE) Comment
Join AD X
Instigated from B6200 over the Management interface but it requires a manual entry in DNS for
the service set first
Media Server (Deathstar) X X MMC permissions are best set from the Media Server
on the 10GbE (data)interface
1. Use the StoreOnce GUI to identify the Network Path for the share and the Service Set on which it resides.
Note: To allow access to NetBackup75NAStarget on Service Set 3 we have to present Service Set 3 to Active Directory. Before we do this, please make a DNS entry for the name of the CIFS server being presented on Service Set 3 (in our case hp4a9273bd8c-3, which can be ascertained by looking at the NAS page).
2. You can now use Windows Explorer on Deathstar to check that it has access to this share.
3. Select NAS in the StoreOnce Navigator, select Service Set 3 (where the NetBackup75NAStarget share resides) and on the CIFS Server tab choose AD from the Authentication drop down list. Click Update.
5. If the domain controller is found, a pop-up box will request credentials of a user with permission to join the domain. Provide credentials (username and password) of a domain user that has permission to add computers to the domain and click Register.
6. If successful, a message is displayed and the CIFS Server screen is displayed.
7. If we now use the StoreOnce GUI to look at the NAS share from which we wish to import, we can see that the authentication has been set to Active Directory.
8. Working on the Media Server, Deathstar, connected to the 10GbE network, set the access permissions using the MMC snap-in, Shared Folders. Type MMC in the “search programs and files” dialog box on the Windows Start menu.
9. Click File -Add/Remove snap-in. Enter the IP Address of Service Set 3, where NetBackup75NAStarget resides and click Finish.
10. Click OK, and the MMC returns to the console – you can see the shares presented. Right click on NetBackup75NAStarget and click Properties.
Note: The share is still read only at this time. If write access to the share is required, you must first break (delete) the NAS replication mappings).
In the early configuration of this NetBackup test environment we ensured all the NetBackup services use the Domain Administrator logon – so the import we are about to implement will now work.
Making the Master Server aware of the replicated shares
To make the Master Server ( NetBackup Catalog) aware of these images so that we can recover from them, we must use the Import Images wizard once again.
1. Proceed as with VTL, using the Import Images wizard from the main NetBackup Administration Console. Select Deathstar as the media reader, but this time the Image type is Disk (not Tape) because we are dealing with a NAS share. The disk type should be set to Basic Disk, as shown below.
2. Enter the path from which the images are to be imported. This is the Network Path identified earlier on the StoreOnce GUI.
3. Click Next and OK.
4. Phase 1 of the import process starts and can be monitored either in the Activity monitor or on the Results window when selecting the Catalog feature in the left-hand navigation pane.
Note how it is recreating image information for ALL the backups present on the replicated NAS share.
5. Now search the Catalog for the entries it has just created. Leave the Media Server set to <All Media Servers> and Path set to <All> and click Search Now.
6. On the Results page select the latest Full backup (highlighted), right click and then click Import.
Recovering the image to the Disaster Recovery server, Valiant
As previously, the server controlling the restore will be the Master Server (Saratoga); the source client is heartofgold (the original backup client on the primary site), and the destination client is now valiant on the secondary site.
The NetBackup Catalog contains: the previous StoreOnceVTL policy backup of heartofgold, which we have already restored; the StoreOnceNAS policy, which we are about to restore; and the StoreOnceCatalyst policy, which we will restore next and which already exists because of the unique features of OST/Catalyst replication being tracked by the Catalog.
1. Start the NetBackup 7.5 Client.
2. Click File - Specify NetBackup machines and Policy Type (see below. All relevant backups will be displayed in the Backup, Archive and Restore window
3. Select the StoreOnceNAS Policy and the last full backup that we have just imported. Click the start restore icon in the left-hand navigation pane. Select valiant as the Destination client for restore and click OK.
5. Monitor the restore in the Activity monitor.
6. The NAS share restore completes successfully!
StoreOnceCatalyst policy restores
For the final part of DR scenario 1 we will restore from the StoreOnceCatalyst policy and the replicated Catalyst Store residing on the HP StoreOnce Backup system, B6200B, at the Secondary Data Center. This will demonstrate one of the key features of OST/Catalyst integration – Catalog awareness of all copies of data, on both B6200A at the Primary Data Center and B6200B at the Secondary Data Center. The Disk Type for the catalog Search is OpenStorage(hp-StoreOnceCatalyst) .
In this scenario the StoreOnceCatalyst policy has a retention period on the first copy of the backup of two weeks and an extended retention period on the second copy of three months. This means that, at the point of recovery, the first copy has already expired and been cleaned up by the Image Cleanup job within Symantec NetBackup ; when we search the Catalog we can only see Copy 2 of the backup, as shown below.
1. Start the NetBackup Client for restores. As you can see, the Catalog already has entries for the StoreOnceCatalyst policy type on the HP StoreOnce Backup system, B6200B. Proceed as previously. Click File - Specify NetBackup machines and Policy Type (see below). All relevant backups will be displayed in the Backup, Archive and Restore window
This also highlights another benefit of the OST/Catalyst integration; we can set different retention periods for the different copies and this allows for extended retention periods on the DR site. When we originally set up the Storage Lifecycle policy for StoreOnceCatalyst the Copy retention period was set to three months whereas the original backup was set to two weeks.
2. Now simply start the restore for the latest full backup of policy type StoreOnceCatalyst, as shown. The backup will be read by Media Server, Deathstar, and written to Disaster Recovery Server, Valiant.
3. Initially the restore FAILS because the StoreOnce Catalyst Store, NetBackup75reptarget (on B6200B), that we initially configured could only be accessed by the “heartofgold” Media Server.
4. We must allow more Media Servers (like Deathstar) to access the copy of the B6200A Catalyst Store that exists on B6200B. Click Credentials in the left-hand navigation pane of the NetBackup Administration Console.
5. Select the Storage Server b6200ss3.nearline.local that contains the OST/Catalyst Store, NetBackup75reptarget, (where our Disaster Recovery data resides). A list of Media Servers allowed to access b6200ss3.nearline.local is displayed. Currently, it is set to allow heartofgold access only. Tick “deathstar”to define it as another Media Server that is allowed access to Storage Server B6200ss3.
6. Click OK and Apply. 7. Repeat the restore process.
The HP StoreOnce Catalyst restore is successful, easy to implement and takes place in record time.
Scenario 1 conclusions
This example, Scanario 1, illustrates that the VTL and NAS share Disaster Recovery processes function satisfactorily, but they are somewhat cumbersome. And, importing backup images can be time consuming depending on the volume of data to be imported.
By contrast the OST/Catalyst Disaster Recovery process is simple, coherent and easy to use. It is a definite step forward in Disaster Recovery planning.
Scenario 2 – Medium term loss of backup facilities at the Primary Site
This scenario illustrates how the HP StoreOnce Catalyst store provides further advantages in a Disaster Recovery scenario because it allows cross-site backup over a low bandwidth link. Because the Media Server “heartofgold” has carried out all the heavy processing work involved in deduplication, only unique data is transferred across site.
Figure 5: Configuration for Disaster Recovery Scenario 2
We shall now illustrate how a simple change to the Backup Policy/Storage Unit group can be used to redirect low-bandwidth backups occurring at the Primary Data Center to the backup device, called B6200B Catalyst Store, at the Secondary Data Center. This simple exercise shows a simple, but effective, solution to a DR scenario where the B6200A Catalyst Store backup device at the Primary Data Center has an extended outage and there is the possibility of a day’s backups being postponed until the B6200A Catalyst Store device is repaired and put back online again. There are two options:
1. Create a new Catalyst store at the Secondary Data Center but this requires reseeding and extra capacity to be sized on the HP StoreOnce Backup system at the Secondary Data Center.
2. Reuse the existing replicated Catalyst store so there is more data in common and reduced seeding is required.
Our example uses the second option. We shall first create a new Storage Unit Group, called CatalystStorageGroup, and then create a new Storage Life Policy, called PDCB6200Down, for the backup operation.
Creating the Storage Unit Group
Symantec NetBackup Storage Lifecycle policies allow the use of “Storage Unit Groups” – this is in order to provide an automated, optimal use of all storage units available as backup targets to NetBackup, according to established rules.
1. From the left-hand menu select Storage Unit Groups,. Right click and create a Storage Unit Group such as CatalystStorageGroup (shown below). In this Storage Unit Group we will place our Catalyst Store at the Primary Data Center (NetBackup75LowbandwidthStore) and our replication Catalyst Store at the Secondary Data Center (NetBackup75reptarget).
2. The Storage Unit Group is then specified in the Storage Lifecycle Policy, which gives us the option of automating what should happen if a member of a storage group is unavailable. There are two options:
Under normal conditions we would specify the Primary Data Center Catalyst Store as the highest priority Storage Unit item in the Storage Group and set the criteria for Storage Unit selection as “Failover.” This means that if NetBackup75LowbandwidthStore is unavailable because of a major issue at the Primary Data Center, the backup can be automatically redirected to
NetBackupreptarget Storage Unit at the Secondary Data Center, which is already “seeded.” There should be no major impact to
Creating the Storage Life Policy
We will create a new Storage Lifecycle Policy (called PDCB6200Down) and, instead of selecting a single storage unit as before, we will choose the Storage Unit Group, called CatalystStorageGroup , that we previously created.
1. Right click on Storage Lifecycle policies and then click New Storage Lifecycle Policy.
2. Create a new SLP called PDCB6200Down and in the Backup operation, instead of selecting a single storage unit, select the storage unit group called CatalystStorageGroup that we just created.
3. Click OK.
4. Select the new Storage Lifecycle Policy, PDCB6200Down, in our Backup Policy, StoreOnceCatalyst. Right click the Policy,
in the Activity report below.
6. We can also monitor the activity on the StoreOnce GUI, by selecting Service Set 3 and looking at the StoreOnce Catalyst graphs, as shown below.
7. Before we reran the backup we performed our usual “changedata” script to change 10% of the data in 10% of the files and add 10% new data – to simulate real world activity. Notice if we look at the Data Jobs tab for NetBackup75reptarget below, which is now the source for the backup, we are still getting 99.1% bandwidth saving because the B6200B is still “in-sync” with what was previously backed up to the B6200A proving that this is indeed a proven practical approach.
This is good news for the customer because the re-direction of backups to B6200B on the Secondary Data Center has been easy and has not required high bandwidth for a cross-site link – a win-win for the user.
The cross-site backup completes successfully!
A variation on this approach would be if a customer has a requirement for fast offsite SLAs where backups have to be offsite in a small period of time. In this case the following approach could be used.
Backup cross site using Catalyst Low bandwith backup to the HP StoreOnce Backup system, B6200B, at the Secondary Data Center – this then meets requirement for fast offsite SLAs
B6200B at the Secondary Data Center could then replicate back to B6200A at the Primary Data Center in order to provide a local copy at the Primary Data Center for local recovery.
Scenario 3: Restoration of a NEW Primary site from the Secondary site
This scenario illustrates how to restore a new primary site from the secondary site simply by changing a Storage Lifecycle Policy.
The scenario here is where the Primary Data Center has been down for several weeks and the new production servers at the Secondary Data Center have been regularly backed up to the B6200B at the Secondary Data Center. We now want to make the Primary Data Center the “Master” data centre once again.
After the Primary Data Center is rebuilt, by creating another Storage Lifecycle policy we can simply convert the NetBackup75 Catalyst Store to become the replication target for low bandwidth replication from the Secondary Data Center to the newly rebuilt Primary Data Center. The final part of the process – re-installing the Application servers again at the Primary Data Center – will follow the process identified earlier using Scenario1.
This scenario illustrates how OST/Catalyst Stores make it so much easier to manipulate sources and targets!
Figure 6: Configuration for Disaster Recovery Scenario 3
1. To simulate B6200A being a “new” appliance in this scenario we will first delete the backup items in the Catalyst store on B6200A. Select Catalyst Store NetBackup75 on Service Set 1 (Primary Data Center), and display the Item Summary tab. Every NetBackup job creates two entries in the item list: an HDR file and the Backup data file. Delete them both using the Delete button, as shown.
2. The main difference to DR Scenario 2 is that we now have to change the Media Server backing up to B6200B to be “Deathstar” rather than “Heartofgold” – to simulate this we take the Media Server “heartofgold” offline.
3. Create another Storage Life Cycle Policy with backup to B6200B at 2 weeks’ retention and duplicate to B6200A with 3 months’ retention, as shown below. Call the new SLP B6200BtoB6200AReplication.
4. Edit the original StoreOnceCatalyst Backup policy to incorporate the new Storage Lifecycle policy B6200BtoB6200AReplication, as shown below.
5. Change the “Client” from “Heartofgold” to “Valiant” in the StoreOnce Catalyst Policy Client tab.
6. Change the data set to the 2GB_Dataset now residing on Valiant in the Backup Selections tab.
7. Run the Policy StoreOnceCatalyst in Manual Backup mode as previously. The backup job of Valiant to the B6200B Catalyst Store completes successfully.
8. Thirty minutes later the duplication (optimized copy) job tries to replicate the data from the NetBackup75reptarget Catalyst Store to the
NetBackup75 Catalyst Store on B6200A at the Primary Data Center but fails (see screenshot below). The reason it fails is because Media
Server “deathstar” has no connectivity to the Storage Unit NetBackup75Lowbandwidthstore.
9. To resolve this, select the Credentials section in the left-hand navigation pane of the NetBackup Administration Console and choose the B6200ss1 Storage Server ( B6200A). When we originally set up the B6200SS1 Storage Server we only allowed Media Server
“heartofgold” access to it. Now, in DR Scenario 3, it is the Media Server “Deathstar” that must send the OST commands to B62200ss3 and B6200ss1 to implement the optimized copy (duplicate functionality in the NetBackup Storage Lifecycle policy). Tick the checkbox against the “Deathstar” Media Server to allow it to access the B6200ss1 Storage Server.
10. Note how the duplicate job did not fail, it kept on trying. When the Media Server, Deathstar, was given access on 04/09/2102 at 08:25 the duplication part of the Storage Lifecycle Policy ran to completion. Also note on the screenshot below the regular “Image Cleanup” jobs that run to automatically remove expired backup images – including backups to HP StoreOnce Catalyst.
11. Notice if we look at the Inbound Copy Jobs tab for Catalyst Store NetBackup75 on the B6200A GUI we can see the duplication part of the Storage Lifecycle Policy B6200BtoB6200AReplication. As this store had all items removed (simulating a new B6200 at the Primary Data Center being installed after the disaster), the very first duplication (optimized copy) had to perform a “seeding process” and this is shown in the bandwidth saving of only 16.7%. Subsequent “duplications” under the Storage Lifecycle Policy would happen much faster
and have much higher bandwidth savings because the vast majority of the data would already reside on B6200A.
Summary
OST/StoreOnce Catalyst technology has vastly improved the way Disaster Recovery can be implemented by:
Making disaster recovery SLAs much faster because no imports are required and all copies of data are known to the NetBackup Catalog. This is described in detail in Scenario 1 –Total loss of Primary site
Utilizing Low Bandwidth backup techniques to Catalyst Stores to ensure cross-site backup in the event of backup appliance outages at the Primary Data Center. This can be implemented in a cost-effective manner by re-purposing a replication target for primary data site backups.. A variation on this scenario also allows for faster offsite SLAs to be met. For more details see Scenario 2 – Medium term loss of backup facilities at the Primary Site
Simple and easy “site role reversal” when recovering from a complete site rebuild. For details see Scenario 3: Restoration of a NEW Primary site from the Secondary site.
The additional cost of Enterprise Disk Licenses for NetBackup and Catalyst Store Licenses for the HP StoreOnce B6200 Backup system are cost-effective and well worth the investment for the peace of mind they give in accelerating the recovery process when a Disaster strikes!
51
51
For more informationHP StoreOnce VTL, CIFS share and OST/Catalyst integration guide with Symantec NetBackup 7.5, EJ022-90953 To read more about HP StoreOnce go to: http://www.hp.com/go/StoreOnce
Call to action hp.com/go/storeonce OST 2.0 downloads
Get connected
hp.com/go/getconnected
Current HP driver, support, and security alerts delivered directly to your desktop
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Symantec and NetBackup are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
