Linux Backups. Russell Adams Linux Backups p. 1

(1)

Linux Backups

Russell Adams <[email protected]>

(2)

Linux Backup

• Discuss a variety of backup software &

methods

• Recommendations for backup

• HOWTO for Rsnapshot

(3)

Shameless Self Promotion

• Principal Consultant @ Adams Information

Services LLC

• http://www.adamsinfoserv.com/

• HLUG member since 2002

• IBM CATE

• Provides Nationwide services and

administration for IBM AIX, pSeries, SAN Storage, HACMP

• Go Debian!

(4)

Basic Recommendations

• Use separate drives/media for data and

backup

• Use RAID or other redundancy for all disks

where possible

• Stop open applications during a backup (ie:

VMWare, SQL) to prevent open files issues and backup corruption

• Always have a recent full backup • Keep a recent offsite copy

(5)

Basic Recommendations

TEST YOUR BACKUPS

(6)

Backup Software

• Variety of tools & full products for Linux • Only covering file level backup here

(7)

Features / Criteria

• Tool / Product • Tool

• A tool works as a standalone command

that performs a single function.

• Product

• Most products perform multiple functions

and have a user interface. They are

typically more complex than a standalone tool.

(8)

Features / Criteria

• Ease of use • Learning curve • Backup • Restore • Backup Type • Full • Incremental

(9)

Features / Criteria

• Media • Disk • Tape • Indexing • Remote Backup • Compression • Encryption • Automation Linux Backups – p. 9

(10)

Software

• cp • tar • rsync • Duplicity • rsnapshot • AMANDA • BackupPC • k3b • Mondo Rescue • mkcdrec

(11)

cp

• Standalone tool

• Copy files to external drive/partition

• Easy to use, backup and restore use the

same command

• Full backups, can perform incremental

updates

• Disk media only

• No indexing, must be manually maintained

(12)

cp

• Optional remote (ie: scp)

• No compression, encryption

(13)

cp

• Example

• cp -a src dest

• Recursively copies src to dest,

preserves owner/group/time/symlinks

• cp -au src dest

• As above, but only copies items that have

changed (ie: incremental)

(14)

tar

• Standalone tool, Unix standard for archival,

competes with cpio

• Learning curve is mild • Complex flags

• Restores are more complex than backups

• Supports both full and incremental backups • Can write to files on disk or tape

(15)

tar

• Manual indexing, incremental indexes are

required but file specific

• Remote backups supported via remote tape

• Full compression, gzip & bzip2 • No encryption

• Automation requires manual scripting & cron

(16)

tar

• Example

• tar -zcvpf dest.tar.gz src

• Create an archive of files in src,

preserving permissions, with compression.

• tar -g dest.dat -zcvpf

dest.tar.gz src

• As above, but save an index file for

(17)

tar

• Example

• tar -C dest -zxvpf src.tar.gz

• Restore from archive to dest, preserving

permissions

(18)

rsync

• Standalone tool with fantastic flexibility

• Easy to use, can replace cp command

• Moderate learning curve, many options • Backup & Restore can use the same

command

• Full backups with fast incremental updates • Writes to disk files only

(19)

rsync

• Remote backup fully supported, native and

over scp

• Common tool for maintaining remote

mirrors

• Can optionally compress network traffic, no

file compression

• No encryption, when using scp network

traffic is encrypted

(20)

rsync

• Example

• rsync -av src dest

• Copies src to dest, preserving

owner/group/permission/timestamp

• rsync -av src user@host:dest

(21)

rsync

• Complex Example

r s y n c ₋₋verbose ₋₋s t a t s ₋₋r e c u r s i v e ₋₋perms \ −−t i m e s −−d e l e t e −−l i n k s \

−−max−d e l e t e =50 −−compress −−i t e m i z e−changes \ −−e x c l u d e rladams / . m o z i l l a / f i r e f o x /∗/ Cache \ −−e x c l u d e rladams / home \

−−e x c l u d e rladams / music \ −−e x c l u d e rladams / tmp \

−−l o g−f i l e = /home / rladams / . rsync_home . l o g \ / home / backup home : / backup / t h i n k p a d /

(22)

Duplicity

• CLI Product

• Key feature is creating remote backups on

untrusted storage

• Moderate learning curve

• Backup and restore use similar commands

(23)

Duplicity

• Only writes to disk or remote host • Internally indexed for incrementals

• Remote backup via SSH

• Full compression

• Encrypted via GnuPG

(24)

rsnapshot

• Full featured tool for creating backups • Fantastic storage algorithm

• Easy to use

• Backup to a directory tree

• Restore is simply copying files back into

place

• Full backup with multiple incrementals • Disk only

(25)

rsnapshot

• Indexed by iteration

• Pull remote backups supported via “rsync” &

“scp”

• No compression or encryption

• Automation is easy • One line cron job • Setup config file

(26)

rsnapshot

• Example

• Covered in depth in the HOWTO

• rsnapshot weekly

• Take an incremental backup to weekly

(27)

AMANDA

• Product

• Server side, multiple clients

• Intelligent backup scheduling logic & tape

handling

• Key features are disk spooling and tape

indexing & library automation

(28)

AMANDA

• Steep learning curve • CLI Only

• Backups are automated

• Restores are manual via ftp-like interface • Supports repeated full backups and

(29)

AMANDA

• Writes to tape, uses local disk for spooling

backups (fast!)

• Complex indexing

• Supports remote backup and compression

• Optional encryption

• Fully automated on the server side

(30)

BackupPC

• Product

• Server side, multiple clients

• Intelligent backup scheduling logic

• Key features are user interface & rsnapshot

(31)

BackupPC

• Moderate learning curve

• Fantastic web interface with multiple user

support & security

• Backups are easy to setup • Restores via web interface

• File explorer, download files from browser • Zip file download

• Push to host

(32)

BackupPC

• Supports full backup with multiple

incrementals

• Disk only

• Complete indexing

• Optional on-disk compression

• Encryption over network using SSH, no file

encryption

• Complete automation, jobs created &

(33)

BackupPC

(34)

(35)

BackupPC

(36)

(37)

BackupPC

(38)

(39)

Other

• k3b

• Deserves a mention for a quick method to

backup files & directories to CDROM.

• Mondo Rescue

• Mondo Rescue takes a “bare metal”

backup for full system restore.

• Popular with Redhat

• mkcdrec

• Another “bare metal” backup tool

(40)

Rsnapshot Storage Algorithm

• Rsnapshot uses hard links in a creative

fashion to minimize the amount of storage required for multiple backups

• Total size is the total for one full backup and

only changes for as many incrementals that are kept

(41)

Sample Disk Usage

46G / backup / d a i l y . 0 / 263M / backup / d a i l y . 1 / 269M / backup / d a i l y . 2 / 248M / backup / d a i l y . 3 / 8 . 6G / backup / d a i l y . 4 / 3 . 1G / backup / d a i l y . 5 / 252M / backup / d a i l y . 6 / 252M / backup / d a i l y . 7 / 247M / backup / d a i l y . 8 / 253M / backup / d a i l y . 9 / 59G t o t a l Linux Backups – p. 41

(42)

Files & Linking

Legend File Inode Symlink Regular File File_A.txt Inode 1

(43)

Symlinks vs Hard Links

Symlinked File File_A.txt Inode 1 File_B.txt Hardlinked File File_A.txt Inode 1 File_B.txt Linux Backups – p. 43

(44)

Rsnapshot Backup

Cloning new backup

Time.0 Time.1

File_A File_A

File_B File_B

(45)

Rsnapshot Backup

Update clone via rsync

Time.0 Time.1 File_A File_A File_B File_C File_C File_B Linux Backups – p. 45

(46)

Rsnapshot Backup

Backup Repository on Disk

Time.0 Time.1 Time.2 Time.3

File_A File_A File_B File_B File_C File_A File_C File_C File_A File_B File_B File_C

(47)

HOWTO

• Backup script

• Udev rule for automatic automount & backup • Cron job

• Rsnapshot.conf

(48)

Backup.sh

# ! / b i n / sh ₋ex

[ ₋f / . nobackup ] && e x i t

exec > / tmp / backup . l o g 2>&1

mount / mnt / backup r s n a p s h o t weekly sync

(49)

Udev Rule /etc/udev/rules.d/10-my.rules

KERNEL==" sd ? 1 " , SUBSYSTEMS==" usb " , ATTRS{ i d P r o d u c t } = = " 0 7 0 2 " , \ ATTRS{ idVendor }=="05 e3 " , SYMLINK=" backuphd " , \

RUN= " / b i n / sh / r o o t / backup . sh "

(50)

Sample Cron Job

(51)

Rsnapshot.conf

c o n f i g _ v e r s i o n 1 . 2 s n a p s h o t _ r o o t / mnt / backup / snapshots / n o _ c r e a t e _ r o o t 1 cmd_cp / b i n / cp cmd_rm / b i n / rm cmd_rsync / u s r / b i n / r s y n c cmd_logger / u s r / b i n / l o g g e r i n t e r v a l weekly 5 verbose 5 l o g l e v e l 5 l o g f i l e / tmp / r s n a p s h o t . l o g l o c k f i l e / v a r / run / r s n a p s h o t . p i d backup / e t c t h i n k p a d / backup / home t h i n k p a d / Linux Backups – p. 51

(52)