QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
1
Overview
GBS Hybrid Backup server
The Generic Backup Solution (GBS) offers a Hybrid Backup server that is capable of backing up Virtual systems, Non-virtual systems, SAN and network equipment.
The Hybrid GBS backup server is a hardware oriented backup solution that will make independent backups outside the virtual environment (including offsite backup possibility).
Main goal is to provide a full recovery mechanism for a disaster in the virtual environment.
The Hybrid Backup server is a non-virtual backup server (management station), based on HP DL380/BL460 hardware platform (RHEL6) with additional data disks for backup storage.
The Hybrid Backup server has the same general architecture and functionality as the Cloud, Virual and Legacy backup server, extended with additional functionality for the backup of VMware Virtual Machines and ESXi (Hypervisor) configuration backup. The Hybrid GBS backup server offers the following backup functionality:
Centralized backup to backup all elements in the customer enterprise solution onto a single backup disk repository. Simplifies interfacing to 2nd level (tape,cloud) backup.
Full system backup (rsync based), for virtual and non-virtual systems.
File based backup, allows easy browsing and restore of files and directories.
Disaster recovery for (non) virtual systems (Linux, Solaris, OpenVMS).
Application data backup and database backup.
VMware image backup of virtual machines (for disaster recovery).
VMware Hypervisor backup of ESXI configuration for several ESXi nodes.
Backup of network elements (configuration files).
Backup to external disk storage/SAN storage.
Maintain backup history, based on “rsnapshot” tooling and hardlink based deduplication.
Tape backup; Direct interfacing to Tape autoloader. Bacula option (open source, license free) or EMC networker.
Internal backup storage capacity up to 72TB.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
2
Functionality (general)
Advantages of a Hybrid Backup server:
The Hybrid Backup server has advantages compared to vendor specific mechanism’s.
Vendors usually offer backup mechanisms that still need to be configured and tested for each application.
GBS will fully configure, deploy and test the backup procedures. These have already been tested and documented for various Linux distributions and applications, it therefore saves a lot of work in each messaging project, as the backup & restore part in the project can be minimized.
GBS can interface with Vendor specific backup mechanisms, and will act in that case as a backup gateway to the applications. Other GBS advantages:
Backups are independent from virtual infrastructure, backups can be restored (redeployed) on different virtual infrastructure (KVM vs VMware) or on bare metal hardware. So the backup image can be used to migrate a VM to a non-virtual server.
Can be deployed on an existing NGP management stations.
Direct interfacing to hardware and disk I/O, giving higher performance and capacity.
Backups are stored outside the virtual infrastructure, which means that the backups are still available in case of a disastrous event in the virtual environment.
Backups can be used to restore the virtual infrastructure.
Hypervisor backup to restore the VMware ESXi virtual host node(s) after a disaster.
Can be used in an existing GBS solution where virtualization is introduced (Hybrid) to back up virtual machines.
Centralized Backup
The Generic Backup Solution delivers a universal backup mechanism to backup all elements in the customer enterprise solution onto one centralized backup disk repository.
Backups are easy to find, accessible in a central place in your cloud solution, managed (and thus protected) from within your own virtual environment. GBS is an easy backup service for operations as it eliminates the hassle of tape backup.
The centralized backup will simplify interfacing to various 3rd party backup solutions (e.g different customer main backup systems), because there is only one repository to backup and one interface to arrange.
The backup images are accessible as regular files, which make it possible to edit files, search for configuration changes, and modify images before the restore.
Permanent disk backup also eliminates the additional cost for tape hardware and licenses.
GBS contains backup tooling to back up:
Linux/Solaris/VMS operating systems.
Cisco, HP and F5 network equipment configurations.
HP EVA/3PAR storage configuration and data.
Data repositories on local or EVA disk.
Databases Oracle/Berkeley/Informix.
Backup Server PXE Boot Server
Customer specific Application server (RHEL)
LDAP Directory server (RHEL)
Mail server (RHEL)
Network Components System B ackup Ne tw or k_ Ba ck up Legato/ Veritas Tape Backup (Optional) System Backup Syst em B acku p
EVA Disk Array – Mirrorclone Mechanism IMAP LDAP OpenVMS server BA CK UP _I MA GE Data Backup Permanent Temporary HP EVA Backu p
Daily backup
The basic GBS backup mechanism will download all backups from application nodes and network elements on a daily basis (at night). The Daily backup will include:
System backups to download the full directory structure of the application node (system disk only).
Data backups to download data stores/databases outside the application’s system disk.
Network backups of routers, switches, load balancers etc.
VMware backups to clone virtual machines (for disaster recovery) and ESXi node configuration.
The daily backup is a full/incremental backup that is updated each night. Therefore it does not contain backup history. The system backup also includes a PXE-boot based disaster recovery mechanism.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
3
Functionality (general)
Permanent Backup to Disk for History
In order to preserve historical backup information, GBS offers tooling to copy the daily backup to permanent storage (on local disk). The permanent disk backup stores several full copies of the daily backup:
7 daily backups per week (max).
4 weekly backups per month (max).
3 -12 monthly backups per year (configurable). Default is 3 monthly backups.
In the permanent backup a static file is stored once on disk and is “hardlinked” from each permanent backup to save diskspace. Permanent backup to SAN storage is optional.
Retention time of permanent backup is determined by the amount of diskspace available vs the amount of data that needs to be stored. The number of permanent backups will be tuned to match the disk space available.
Application integration
For the data backup of application specific data repositories and databases GBS, will be configured according to the application backup procedures. For application databases, the GBS backup server will typically trigger database specific extract functions to assure a consistent database backup.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
4
Backup Mechanisms
Mechanisms: System/Data/ Network backup
GBS uses 4 mechanism’s to accomplish the backups from different elements in the customers Enterprise Solution:
“System backup” mechanism to copy/synchronize the system disk content to the backup server’s central repository. For the system backup, a GBS agent RPM needs to be installed on the application node.
“Data backup” mechanism will copy the content of specific data stores (outside the system disk) to the GBS backup server’s central repository.
As the data backup mechanism only uses plain “RSYNC”, no agent software needs to be installed on the application node.
“Network backup” will dump the switch/router configuration files to the backup server’s central repository. Usually, the running/start-up config of the switch/router is copied to the backup server. In some cases, other config files (e.g. ITP routing tables) are included.
A “local backup” mechanism is specific for each application, in order to prepare databases for consistent back up.
Disaster recovery (Linux, Solaris, VMS, Virtual Machines)
GBS includes a Linux based disaster recovery mechanism based on Open Source “SystemImager” tooling. SystemImager creates a mirror copy of the whole backup client system disk onto the backup server.
Also the file system and partition info of the system disk is captured so the system disk can be recreated from scratch in case of a disaster.
The restore is fully automated and does not require knowledge on how to deploy the application.
The disaster recovery mechanism uses the (PXE) network boot to boot from the backup server and to restore the system disk, typically within 30 minutes. The Disaster Restore can be executed remotely and does not require onsite presence.
In a virtualized solution, the SystemImager Disaster Recovery mechanism offers filespace advantages over the VMware VMDR mechanism. It stores only the actual filespace, and (big) repositories can be excluded to limit required storage.
The disaster recovery mechanism is hardware independent and also independent of the virtual environment.
Therefore it can be used in different virtual environments (VMware, KVM) or even to migrate a virtual machine between different virtual environments. It even allows migration of virtual to non-virtual machines.
For Sun Solaris there is a similar tool available based on FLAR archive.
For OpenVMS a disaster recovery mechanism is based on the VMS backup utility.
For VMware, an additional disaster recovery (VMDR) is available, based on a VM cloning mechanism.
Disaster recovery for Virtual Machines (VMware ESXi)
GBS includes a Virtual Machine Disaster Recovery (VMDR) mechanism to recover VMware virtual machines on ESXi. The mechanism is based on Open Source “XSIBACKUP” tooling and RSYNC data transfer.
XSIbackup creates an online (“hot”) backup of the virtual machines, copying the VMware vmdk files in a consistent way using VMware snapshots.
VMware “vmdk” files are copied to the central backup repository using RSYNC data transfer. Virtual Machines can be restored by using the vmdk files from (last night’s) backup.
Advantage over the SystemImager Disaster recovery is that the VMDR mechanism is not limited to Linux guest OS.
OVF template deployment/recovery for Virtual Machines (Cold backup)
GBS includes an OVF template backup/recovery mechanism to recover VMware virtual machines on ESXi.
The OVFtool mechanism is integrated with the GBS backup server and allows one to import/export virtual machines as an OVF template. The backup repository includes a container for storing the OVF templates per virtual machine.
OVF export can only be executed on a power-down virtual machine and is therefore considered as a “cold backup” mechanism. Because of the “cold backup” nature of the OVF mechanism, this backup will usually be executed only once after the deployment of the virtual machine. Therefore it needs to be complemented with a daily system backup to capture the latest changes since the OVF was created.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
5
Backup staging
Staged Backup
The Generic Backup Solution (GBS) is based on the concept of “staged backup” where all backups are collected into a central directory in several stages.
Generic Backup Solution (GBS) delivers a universal backup mechanism to back up all elements in the backup solution onto one centralized backup disk repository:
Stage 0 (local backup) for extracting application specific database content.
Stage 1 (temporary backup) on the backup server for the daily backup.
Stage 2 (permanent backup) where the backup is subsequently (optional) streamed onto tape or disk. The permanent backup maintains historical backup data.
For the permanent backup to disk (SAN) open source tooling is used to offer a full backup solution to the customer without the need for EMC networker/ Symantec Netbackup licenses or knowhow.
Stage 0: Backup preparation
When the backup client has a database that needs to be backed up, a consistent copy of the database needs to be prepared on the local node prior to backup collection.
In stage 0 these kinds of application specific backup issues are handled. E.g. for an OpenLDAP directory server a utility must be run to create a consistent backup of the LDAP database.
For each new application the stage 0 aspects need to be considered separately as an integration/project effort.
GBS re-uses application specific backup procedures as much as possible. In the GBS-product integration the product is investigated and data (base) extraction scripts are reused to create a consistent data backup to disk.
Stage 0: Backup encryption
In order to protect sensitive user data, local backup data can be encrypted using GnuPG. Encrypted data files will reside in a separate directory so they can be transferred encrypted to the backup server in stage 1.
Encryption keys will remain on the application node where the encryption is performed.
Stage 1: System Image Backup (image backup for disaster recovery)
GBS implements a system backup mechanism that copies the layout and the content of the system disk to the GBS backup server. This system image will be used for disaster recovery and for restoring individual files/ directories.
The system backup gathers metadata to describe the layout of the system disk, partitions and file systems. So this metadata is used to recreate the system disk in the event of a disaster.
The system backup will be overwritten each night and does not maintain a history (history is stored in the permanent backup).
For Linux based clients the online backups will be gathered on the backup server by using the SystemImager tool; which is an open source tool.
For Solaris based systems (CMS) the FLAR archive function can be used to create a full/ disaster backup. This FLAR archive will be collected on the central backup server.
For OpenVMS servers in general an imaging tool is available to back up a full image of the system disk and transfer that image to the backup server. This (weekly) image backup can be used for disaster recovery and can be extended with a configuration backup script for daily backups (SBR_MENU).
For Network Elements the existing Network backup tooling will be used to collect the backups of all network components.
Stage 1: Data backup
For data that resides outside the system disk (E.g on SAN, external disk, or in a database), GBS has implemented a “Data Backup” mechanism based on open source “rsync” tooling.
The system backup will be overwritten each night and does not maintain a history (that is handled in the permanent backup). The data backup can also be used to run a system backup without a disaster recovery possibility.
Usually the data backup is used as a complementary tool to back up those data directories/databases that are not included in the system backup. The content of the data backup needs to be reviewed for each application.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
6
Backup staging
Stage 1: Network Backup
A network backup script is included in the GBS delivery, which dumps the switch/router configuration files to the /backup/network directory on the backup server. Usually, the running/start-up config of the switch/router is copied over to the backup server. In some cases, other config files (e.g. ITP routing tables) are included. Currently, the network backup supports:
IOS Based Cisco Routers, Switches, Console Routers and CSS Load Balancers.
HP brocade SAN Switches.
HP C7000-gbe2c-L2 switches.
McData Sphereon 4700 Fibre Channel Switch.
Cisco ITP. F5 BIGIP load balancer.
Stage 2: Backup to Disk (Permanent backup)
In order to maintain a daily, weekly and monthly backup history, a separate “permanent” backup repository is maintained on a separate diskgroup on the GBS backup server.
Stage 2: Backup to Tape (Optional)
The collected data on the backup server can be written to tape as an additional level of availability, using EMC networker or Symantec Netbackup or Bacula (preferred).
Stage 2: Offsite backup
GBS includes Bacula server (open source) to send the backups to a remote Bacula Storage Daemon. This Storage daemon will interface with the Tape storage or with disk.
This allows you to put the backups offsite, to protect against a site disaster.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
7
Watchdog, Tape backup and sizing
Watchdog and SNMP Alarming
A GBS watchdog process will run each morning to verify the backup status of the previous night and send an SNMP trap when a backup failure is detected. When no errors are found a “backup success” alarm is sent.
The watchdog will run each morning typically between 9-10AM to alert the operator of the backup status of that night. The watchdog will also check the disk space for the backup repositories.
For backup push clients, the watchdog can check the backup push status (e.g for OpenVMS) by verifying a “backup.complete” file to see if the pushed backups are up to date or outdated.
When no errors are found a “backup success” alarm is send to clear previous error alarms.
Tape backup - general
Tape backup is an option for the GBS Hybrid Backup server, offering an extra level of availability to the permanent disk backup. The GBS backup server can function as a tape client when the tape unit is connected on a remote tape server.
The GBS backup server can function as a tape server when the tape unit is connected to the GBS backup server directly.
In both cases the source of the backups is the backup repository on the GBS server (application nodes do not interface with the tape backup directly).
Tape backup is scheduled in a “second stage” at daytime, after the nightly backup window.
The backup data can be copied to tape during the rest of the day, allowing more time for the tape backups to complete. For Tape Backup GBS offers Bacula (open source) or EMC networker (licensed) as an option.
Bacula is the preferred solution, as it reduces the OPEX required for EMC Networker licensing.
Sizing of the backup repository
For Hybrid Backup servers where the backup storage can be allocated on SAN, the sizing needs to be calculated from the number of backup clients, databases and retention time.
Possibilities to extend Backup repository disk space to allow addition of more backup clients (virtual machines):
Configure disk space for the backup repository on SAN, HP-EVA/3PAR.
Configure additional local data disks in the DL380.
Configure extra disk space on local system disk in the Blade server. Requires (1,2TB system disk PN:718160-B21). GBS backup will calculate sizing requirements for backup projects.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
8
Hardware options
The Hybrid Backup server can be deployed on any HP DL380/385 or BL480 server provided there is enough spare disk capacity to store the backup data. The GBS backup server can be installed (and has been tested) on any RHEL4, RHEL5 or RHEL6 Linux distribution.
The preference for hardware is a DL380 server because it accommodates the largest possible disk storage.
a DL380G6/G7 system can be expanded with 14x1TB data disks.
a DL380G8 system can be expanded with 22x1TB data disks.
a Blade BL480 can be fitted with 1,2TB system disks, to provide a 1TB backup storage.
Hardware options DL380G9:
The Hybrid Backup server can be deployed on any DL380 server, but the following configurations are suggested to create a 24TB or 72TB backup server
DL380G9 (12LFF) with 72TB (12x6TB) data disks:
DL380G9 (24SFF) with 24TB (24x1TB) data disks:
The DL380G9 server has the interesting option to plug the system disk (2xSFF) in the rear harddrive bays. The remaining front harddrive bays can all be used to plug backup disks (12xLFF or 24xSFF).
In addition, the DL380G9 server can be conected to a Tape-autoloader for tape backup and to A HP-SAN system for additional backup storage space.
Tape options:
For tape backup any Ultrium Tape Autoloader or MSL2024 tape unit can be chosen.
However, a 1/8G2 LTO-5 Ultrium 3000 Tape Autoloader is suggested, as it provides 24TB of storage, and a fibre channel interface.
QuickSpecs
GBS_R2.0 Hybrid Backup server
GBS Backup Nederland b.v.
v1.4 - October 2015
[email protected]
9
Deployment and options
Deployment and support
GBS backup company can deploy the basic Hybrid Backup server on request.
Optional functionality can be deployed and configured by GBS Backup company and quoted on request.
A support fee will be charged for each basic Hybrid Backup server that is deployed by the customer or by GBS backup. Additional support will be offered and charged for optional functionality, as agreed in the project proposal.
Optional functionality and documentation
The Hybrid Backup server can be expanded to extend the backup on the following areas:
Backup of non-virtual systems in the same solution/VLAN.
Backup of (non) Virtual system in a second VLAN
Encrypted backups.
Migration to a non-virtual backup server.
Sizing of the backup repository.
Integration with Network Deployment NDS (Cobbler/Koan) to manage network boot.
Tape backup configuration (2nd stage, EMC networker or Bacula)
SAN Snapshot backup to backup databases and repositories on SAN (e.g to backup large Oracle databases for BT application)
SMSC image backup / disaster recovery on a daily/weekly basis