http://xingsolar
http://xingsolar.com/San/Backup/L.com/San/Backup/Legato.htmegato.htm
Melonm Documentation Project
Melonm Documentation Project
Legato in OnePage
Legato in OnePage
Command
Command | | Backup/RestoreBackup/Restore | | StorageStorage | | WindowsWindows | | NMDANMDA | | Daily Jobs Daily Jobs
Command
Command
MMINFO MMINFO•• Based on date/timeBased on date/time
◦
◦ All volumes written on in the past week (not including today):All volumes written on in the past week (not including today):
mminfo -m -t 'last week' mminfo -m -t 'last week'
◦
◦ List backup done one day ago: mminfo -q "savetime > List backup done one day ago: mminfo -q "savetime > 1 day1 day
ago" ago"
◦
◦ Get savesets of SQL group 3 months Get savesets of SQL group 3 months ago: mminfo -av -ot -qago: mminfo -av -ot -q
"savetime
"savetime<<'91 days ago', group='SQL Backups'"'91 days ago', group='SQL Backups'"
◦
◦ Look for saveset between date: mminfo -av -qLook for saveset between date: mminfo -av -q
"client=usuman1,savetime>09/27/08,savetime<09/30/08" "client=usuman1,savetime>09/27/08,savetime<09/30/08"
◦
◦ Look for saveset after date: mminfo -Look for saveset after date: mminfo -av -q "client=usuman1" -tav -q "client=usuman1" -t
"10/28/08" "10/28/08"
◦
◦ Or: mminfo -av Or: mminfo -av -q "client=usuman1,savetime>10/28/08"-q "client=usuman1,savetime>10/28/08" ◦
◦ More complicated examples to query between timesMore complicated examples to query between times
mminfo -s uslgto2 -c uscoredb3 -r mminfo -s uslgto2 -c uscoredb3 -r
"client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "savetime>\"09/30/2008 16:59:00\",
"savetime>\"09/30/2008 16:59:00\", savetime<\"10/01/200savetime<\"10/01/20088 20:59:00\""
20:59:00\""
mminfo -av -s uslgto2 -c uscoredb3 -q
mminfo -av -s uslgto2 -c uscoredb3 -q "savetime>\"09/30/2008"savetime>\"09/30/2008 16:59:00\", savetime<\"10/01/2008 20:59:00\""
16:59:00\", savetime<\"10/01/2008 20:59:00\"" mminfo -s uslgto2 -c uscoredb3 -r
mminfo -s uslgto2 -c uscoredb3 -r
"client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "savetime>\"12/31/2008 20:59:00\",
"savetime>\"12/31/2008 20:59:00\", savetime<\"01/01/200savetime<\"01/01/20099 20:59:00\""
20:59:00\""
◦
mminfo -a -r "volume,%used,pool" -t "-3 days" mminfo -a -r "volume,%used,pool" -t "-3 days" Note: The -t can be used with varioud timeframe. Note: The -t can be used with varioud timeframe. * Relative time:
* Relative time: "-3 second/minute/hourday/week/mont"-3 second/minute/hourday/week/month/year".h/year". * Absolute time: "05/11"
* Absolute time: "05/11"
* Grammar time: "yesterday/last week" * Grammar time: "yesterday/last week"
◦
◦ List saveset in the volume based on expiration date (retentionList saveset in the volume based on expiration date (retention
policy) policy)
mminfo -o e -r
mminfo -o e -r "name,savetime(20),ssbrowse,ssretent" -q"name,savetime(20),ssbrowse,ssretent" -q "volume=XING.Unix.OracleD.0001"
"volume=XING.Unix.OracleD.0001" •• Based on SSIDBased on SSID
◦
◦ List specifics of SSID: mmList specifics of SSID: mminfo -q ssid=120010372 -Vinfo -q ssid=120010372 -V ◦
◦ Search for SSID of Search for SSID of particular client in particular day:particular client in particular day:
mminfo -s uslgto2 -r mminfo -s uslgto2 -r
"client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "client(1),volume(1),barcode(1),ssid(1),name(1),savetime(1)" -q "savetime>\"09/30/2008 20:59:0
"savetime>\"09/30/2008 20:59:00\"" | grep 0\"" | grep uscoredb3 | grepuscoredb3 | grep 09/30/08
09/30/08
•• Based on hostBased on host
◦
◦ mminfo -N <filesystem> -c <hostname> mminfo -N <filesystem> -c <hostname> # list volumes/date# list volumes/date
related to <filesystem> in <hosthame> related to <filesystem> in <hosthame> eg: mminfo -N / usoradb8
eg: mminfo -N / usoradb8
◦
◦ List particular filesystem backup history in Legato: mminfoList particular filesystem backup history in Legato: mminfo
-avot -q name=/obackup3 -avot -q name=/obackup3 •• Based on volume/mediaBased on volume/media
◦
◦ Report contents on media: mminfo -m "Unix.1541"Report contents on media: mminfo -m "Unix.1541" ◦
◦ To display a media report of all To display a media report of all non-full volumes, showing thenon-full volumes, showing the
percent used, pool name, and location of each volume: percent used, pool name, and location of each volume: mminfo -a -r 'volume,%used,pool,location' -q '!full' mminfo -a -r 'volume,%used,pool,location' -q '!full'
◦
◦ List Saveset backed up in one volume: mminfo -av List Saveset backed up in one volume: mminfo -av <vol name><vol name>
•• Tape Usage QueryTape Usage Query
◦
◦ mminfo -avot -r mminfo -avot -r "olabel,labeled,mounts,recycle"olabel,labeled,mounts,recycled" <tape name>d" <tape name> ◦
◦ olabel : the first time the volume was labelledolabel : the first time the volume was labelled ◦
◦ labeled : the most recent time the volume was labelledlabeled : the most recent time the volume was labelled ◦
◦ mounts : number of time the label was readmounts : number of time the label was read ◦
◦ recycled: number of time the tape was relabeledrecycled: number of time the tape was relabeled
Scalar_100 # All volumes in the robat Scalar_100 • mminfo: list details backup time of a particular client
◦ mminfo -q client=ussapp1 -r
'savetime(32),sscreate(32),sumsize,name' -t 'last week' -ot • Saveset summary report: mminfo -X
• Print out weekly report: mminfo -X -t "last week"
• List backup volume/date for file system / in client cesario: mminfo -N / -c cesario
• Tapes used last night (the past 12 hours): mminfo -av -r "volume,%used,pool" -t "-12 hours"
◦ NOTE: Above command using mminfo for tapes used past xx
hours is often used but it is wrong because the date mminfo relied on is the date when the tape starting to be used (creation time), not the date tape actually being accessed. For example, if the tape first being used is one week ago, but consequently
being used since then, it will NOT show up as being used over past few days.
◦ Correct command for this purpose is: mminfo -avq
"volaccess>=01/30/2011 17:59:00" -r volume,%used,pool
• mminfo with "-p" option will print out browse/retention time of savesets.
• mminfo with "-v" option will print out recoverable savesets (not only browsable)
• Query all volumes used for a particular client last night: mminfo -mv -c uscoredb3 -t "-12 hours"
• Locate cloneid of saveid:
◦ mminfo -q ssid=2268782890 -r
"volume,name,cloneid,ssbrowse, ssretent,ssflags,state" • Query to get monthend tapes:
◦ mminfo -av -q "savetime>01/30/2009
17:59:00,savetime<02/01/2009 17:59:00, group=Unix_Prd" | grep ^PW
◦ mminfo -av -q "savetime>01/31/2009
17:59:00,savetime<02/01/2009 17:59:00" -c uscoredb3 | grep ^PW
◦ mminfo -av -c uscoredb3 -q "savetime<2/02/2009
20:59:00,savetime>2/1/2009 20:59:00" -r "volume,pool,ssid,sumsize,name"
• sscomp(20) can give an approximate value of backup time for each saveset
◦ mminfo -ot -q "group='Windows_Exch', savetime<1 day ago" -r
"name,pool,sscomp(20)"
MMINFO -- Saveset Status
"fl" from mminfo output will reveal status of savesets. Here are meanings of various flags:
• Flags for success
◦ cb - complete browsable ( complete backup data on a single
volume)
◦ tb - tail browsable
◦ hb - Header browsable ◦ mb - middle browsable
◦ cr - complete recoverable ( index has expired)
◦ cE - complete erasable ( retention period has been passed)
• Flags for failure
◦ ca - complete aborted ◦ ha,ta,ma,
◦ ci - in progress ( this might not be a failure but the saveset might
be still backing up or hanged)
◦ hi,ti,mi.
• Flags for NDMP
◦ N means NDMP backup (eg, cbNS,hBNS etc.)
NSRJB
• Reset the autochanger: nsrjb -Hv • Perform an inventory: nsrjb -Iv • Perform quick inventory: nsrjb -II
• Change cleaning tape to use 50 more times: nsrjb -U50 -S58 • Command to label slot 3 tape into "Unix" pool: nsrjb -L -f
/dev/rmt/1cbn -S 3 -b "Unix"
• Unload tape from /dev/rmt/0cbn into specific slot: nsrjb -u -f /dev/rmt/0cbn -S 10
/dev/rmt/0cbn XING.Archive.0001 • Re-inventory Library: nsrjb -Iv • Reset the autochanger: nsrjb -H
• Return tape in the drive to where it came from: nsrjb -u -f /dev/rmt/0cbn
• Withdraw volume to access point: nsrjb -w -j rd=uslgto3.melonm.com:HP-MSL1 -P1 -S 30
• Deposit volume from access point (port 1) into specific slot (total 2 ports #1 & #2 in MSL 6060)
◦ nsrjb -d -j rd=uslgto3.melonm.com:HP-MSL1 -P 1 -S 43
MMLOCATE
• List current volumes in the library: mmlocate 'Scalar_100' • List all volumes in particular location/library: mmlocate | grep
Scalar_100 NSRCLONE
• /usr/sbin/nsrclone -v -b <clone pool> <volume name>
• /usr/sbin/nsrclone -s uslgto2.melonm.com -b "XING UnixD" -S 4115098422
• /usr/sbin/nsrclone -s uslgto2.melonm.com -b XING Unix -S `mminfo -N /u4 -c uscoredb3 -t "-24 hours"`
• Clone using storage node:
◦ nsrclone -v -J uslgto3.melonm.com -b "Default Clone"
uslgto3.melonm.com.002
• nsrclone of a saveset out of many copies/clones
◦ nsrclone -v -b "Monthend Unix" -S 2953554712/1225527855 ◦ find clone id: mminfo -q ssid=2953554712 -r
"volume,name,cloneid,ssbrowse,ssretent,ssflags,state"
• nsrclone from file with ssid (make sure enough time between each ssid's clone, since drive needs time to re-initialize)
◦ mminfo -r "ssid" -q "volume=uslgto2.melonm.com.014, !
incomplete"> /tmp/1
◦ nsrclone -b "Default Clone" -S -f /tmp/1
◦ /tmp/1 is the file of all completed ssid in the volume ◦ If not sleep time between ssids, will get error as
"DRIVE_STATUS_NEED_INIT". • nsrclone new features in 7.5.x
◦ nsrclone -v -b "XING Unix OracleD" -S -e now -c uscoredb3 -g
Unix_Oracle
◦ -- -e now: past 24 hours
◦ -- Without -b flag, it will clone to "Default Clone" pool ◦ nsrclone: clone a savegroup from last night: nsrclone -v -b
"XING Windows" -S -e now -g Windows_Usr
◦ nsrclone -v -b "XING Exchange" -S -t "07/01/09 18:59:00" -e
"07/02/09 8:59:00" -c usevs1 -g Windows_Exc
• A script to get required SSID and run nsrclone for them in cron. Download from here.
NSRADMIN
• Be extremely careful about using nsradmin>delete <query> clause. It will delete client as long as <query> is
matched/successfully. Always using "update" is resources need to be updated.
• List related attributes using "echo-->print"
◦ echo "print type: nsr <group>" | nsradmin -i - > <output>
cd/search output file and get related attributes (eg, "autostart" or "last start")
◦ More specific example: echo "print type: nsr client; name:
<clientname>" | nsradmin -i - > <output>
◦ List of resource types can be used with "echo-->print"
combination.
nsradmin> show type name nsradmin> print type: type type name: NSR;
type name: NSR notification; type name: NSR license;
type name: NSR device; type name: NSR stage; type name: NSR group; type name: NSR directive; type name: NSR schedule; type name: NSR policy; type name: NSR client; type name: NSR jukebox;
type name: NSR label; type name: NSR pool;
type name: NSR archive request; type name: NSR migration;
• List all clients and their savesets
◦ Create a text file client_saveset.txt ◦ show name
◦ show save set ◦ show group
◦ print type: nsr client
◦ nsradmin -i client_saveset.txt > <output.txt>
• Update all clients property (update field of storage node)?
◦ /var/tmp/emc/sysname lists all hostname one per line ◦ Script downloads here.
• Migrating client to new server using nsradmin
◦ Generating client property
echo "print type: nsr client; name: <clientname>" | nsradmin i -> client_config
◦ Copy client_config text file to new server
◦ Edit client_config file and update the following property
First line: update "type: NSR client;" to "create type: NSR client;"
Remove number of "client id:"
Update "server:" to new server name
Update other fields non-existing in new server (eg, group name, directives, etc.)
◦ Import client into new server
nsradmin -i client_config
• Disable GSS authentication between client & server
◦ When you get error as "GSS Legato authentication from
<servername> failed", you will need to disable GSS authentication in clients.
◦ nsradmin -p nsrexec ◦ . type: nsrla
◦ show auth methods ◦ print
◦ update auth methods: "0.0.0.0/0,oldauth" ◦ Print
◦ Restart Legato agent afterwards.
How to enable nsrd debugging mode for troubleshooting? • Run date on client side and record time.
• ps -ef | grep nsrd --> get PID of nsrd • dbgcommand -p 26863 debug=9
◦ All debugging information will be written to daemon.log file.
Daemon.log will grow rapidly. • dbgcommand -p 26863 debug=0
◦ Turn debugging off once detail collected.
• Based on recorded time previously, looking through daemon.log for errors.
How to troubleshooting Legato networking problem? • Starting session
• Using "lsof -i TCP" and grep host/nwsrv for connectivity status Other Legato commands
• List license information: nsrlic
• Search all SCSI devices: /etc/LGTOuscsi/inquire • Command line monitoring tool: nsrwatch [-s server] • Scanner: media verifier and index rebuilder
• List Legaot Networker enabler/auth code: /usr/sbin/nsradmin -f /nsr/res/nsr.res
• nsradmin> show name; enabler code; expiration date; auth code • nsradmin> print type: nsr license
• Adding default Legato Administrator into administrators group:
◦ nsraddadmin -u [email protected] ◦ nsraddadmin -u "user=*,host=*"
• Convert daemon.raw into more readable format: nsr_render_log daemon.raw
• Remove client's index: nsrck -R -Y <client name> • Locate tape drives:
• * inquire -Lc
Backup/Restore
How to Backup files in Legato from client side
• save -g 'XING_USSWIFTNET1_archive' -b "XING Archive" -l 0 -v -E -y '08/15/2013' -w '08/15/2013' / /u1 /u2 /u3
• * -g: Save to group XING_USSWIFTNET1_archive • * -b: Use pool XING Archive
• * -l: Level 0 • * -v: Verbose
• * -E: Estimate total size
• * -y: Retention until 08/15/2013 • * -w: Browse until 08/15/2013 • * Backup set: / /u1 /u2 /u3
How to probe/test backup for clients in server side? • Probe group: savegrp -vvv -p SaveGroup
• Probe client: savegrp -vvv -p -c clientname SaveGroup
• Probe with mode debugging level: savegrp -D9 -c clientname SaveGroup
• Actul backup client: savegrp -vvv -p -c clientname SaveGroup • Backup Index only: savegrp -vvv -O -l full -c clientname
SaveGroup
How to fix error "/dev/rmt/4cbn opening: DRIVE_STATUS_NEED_INIT"?
• nsrjb -HEv --> E option will initialize the drive element status • nsrjb -Iev
How to restore files to local directory? /usr/bin/recover
recover > ls (to see files)
recover > add < filename > (add file to restore list)
recover > changetime mm/dd/yy (change restore time of file) recover > versions < filename > (show file versions)
recover > voline < filename > (show volume file is on)
recover > relocate < location > (relocate restore tiles to location) recover > rocover (do the restore)
An interactive session to restore /etc/mail in usuman1 into /recover directory of local server:
• usuman1# recover -c usuman1 -s uslgto2 • Current working directory is /
• recover> relocate /recover
• recover> changetime 'March 09 2009' • recover> cd /etc
• recover> add mail • recover> recover
An example recover session to restore directory in certain date using "versions" keyword. Download here.
How to restore ad hoc files using Legato? • recover -s uslgto2 -d /u6 -S 988276637
/usr/sap/PRD/INT/AP/IN/CHECK_STATEMENT_0121.out • -- /u6 is target directory
• -- 988276637 is SSID
How to exclude certain directories in Windows?
• Create nsr.dir file under above directories where you want to exclude.
• Content: skip: ZyLab_6
How to create pre/post exec script for Legato backup client? • Create related .res file under /nsr/res (eg, Unix_Archive.res) • type: savepnpc;
• precmd: "su - oracle -c '/oracle/dba/bin/start_hot_backup'"; • pstcmd: "su - oracle -c '/oracle/dba/bin/end_hot_backup'"; • timeout: "12:00:00";
• Configure Legato client. Make sure in client property "Apps & Modules" tab, choose "savepnpc" as "Backup Command".
How to restore files in Windows? • Restore savesets
◦ recover -s uslgto2 -d R:\ -S 595232097/1232766305 ◦ * R: mapped destination drive with space
◦ * 595232097/1232766305: ssid/cloneid
• Recover adhoc Windows file:
◦ recover -s uslgto2 -d c:\temp -S 2530483243
W:\PATH1\PATH2\filename.xls
• Get the volume used for the file system: mminfo -c <client name> -N <file system>
• Eg, uslgto2# mminfo -c uscoredb3 -N /u6 • volume client date size level name
• XING.UnixD.0001 uscoredb3 09/25/08 4 B incr /u6 • XING.UnixD.0001 uscoredb3 09/26/08 4 B incr /u6
• XING.UnixD.0001 uscoredb3 09/27/08 9960 MB full /u6 • XING.UnixD.0001 uscoredb3 09/29/08 699 MB incr /u6 • XING.UnixD.0001 uscoredb3 09/30/08 4 B incr /u6
• Get the SSID related to the file system: mminfo -av <volume name> | grep <file system>
• Eg, uslgto2# mminfo -av XING.UnixD.0001 | grep u6 • XING.UnixD.0001 uscoredb3 09/25/08 19:04:45 4 B 2832996617 cb incr /u6 • XING.UnixD.0001 uscoredb3 09/26/08 19:01:03 4 B 4208814506 cb incr /u6 • XING.UnixD.0001 uscoredb3 09/27/08 22:00:08 9960 MB 1692329252 cb full /u6 • XING.UnixD.0001 uscoredb3 09/29/08 18:17:01 699 MB 3420541910 cb incr /u6 • XING.UnixD.0001 uscoredb3 09/30/08 18:09:52 4 B 3504513968 cb incr /u6
• In the restore server, create a script which will restore the full backup & related incr backups:
• Eg, recover -s uslgto2 -d /u6 -S 1692329252 /u6/elvis • recover -s uslgto2 -d /u6 -S 3420541910 /u6/elvis • recover -s uslgto2 -d /u6 -S 3504513968 /u6/elvis
A shell script to check Legato backup time daily. Download from here.
Local directives to exclude certain directoreis in Unix • << /filesystem >>
• action: file.filter
◦ eg, to exclude /olbackup3 & /olbackup4 from backup, create a
file .nsr under / with the following:
◦ << /olbackup3 >> ◦ +skip: .
◦ << /olbackup4 >> ◦ +skip: .
• Note: the plus (+) sign means that it applies to the named directory and all of its subdirectories (i.e. recursive).
How to configure ports for Legato? (ports configuration in firewall)
• In client side:
◦ nsrports, show ports used by NSR client;
◦ rpcinfo -s uslgto2.melonm.com, show ports used for connection ◦ (optional) # nsrports -S 7937-7940 --> restirct ports used in the
client side (these 4 ports are sufficient).
• Open the inbound ports in the firewall for the following 4 ports, so both clients can talk to Legato server/Storage node inside the firewall.
• Usws1 --> ports (7937-7940) --> uslgto2/uslgto3.melonm.com • In the server side, restrict the ports based on the following
fomula:
◦ 12(default)+number ofJukeBox+ 2*number of devices
(tape+aftd) + 5
◦ eg, 12+3 (Jukeboxes)+ 2*12 (number of tape/disk devices)
+5=72
• So the total port will be: 7937-8080 (adding 71 more ports to be on the safe side on top of 72 ports, 7937+72+71)
How does multiplexing work in NW??(would like to increase the amount of drives used on one save group)
• The number of drives used for backup is dependent on different parameters, mainly on drive sessions. In the properties of the drive check the number of target sessions. By default it is 4 so you can increase it to 8. Also verify that the client,storage node and networker server parallelism is more than 8.
• In addition to the above settings check the number of clients in the group. If the backup is targetted to drive with 8 target
sessions first it will start backup of 8 savesets to single drive, if the number of savesets is more than 8 it will then start the
How to backup NDMP data through storage node? • nsrndmp_save -M -T dump -P <storagenode>
◦ -M: backup to storage node. However, traffic from NAS will
come to nwserver first and send to storage node.
◦ -P: using storagenode as proxy host and traffic from NAS will
come to storage node and save on storage node.
◦ -T: type of backup
How to restore NML (Lotus Notes) client through command line?
• nsrnml_remrecov -d "/dparb/restore" -s ryeaxbkm.rye.avon.com -c namail2b -a -X /dparb/data/mail/utrainee.nsf
How to restore NetApps filer with NDMP? -- NetApps view /vol/vol0: java netapp.cmds.jsh -- Recover NDMP file from NetApps
* Only in storage node: uslgto3
* recover -vvv -c usnas1 -s uslgto2.melonm.com recover> cd /vol/vol3_nas1/ToolsRoot/Tools/temp recover> add "file name.pdf"
recover> recover
## 83 minutes for single file from usnas1 Using command line :
---# nsrndmp_recover -c filer_name -s (server) -S ssid -v off -m (filername)::target_dir_path source_dir_path
# Above command can only be used to restore SSID or
directory, not single file. It will restore savesets over browsing policy period.
For Example:
# nsrndmp_recover -c filer2 -s nicodemus -S 964505857 -v off -m burn::/vol/vol2/destination /vol/vol0/user/rkapoor/test
The "-v off" is a new switch added in NW 6.1.3 which
according to man page does not verify the directory path with client file index. This switch can be used to recover data when the Index information is lost or is corrupted.
3433001941 -v off -m usnas2::/vol/vol3_nas2 /vol/vol3_nas2/Users/NSu/Logs
## 110 minutes for single file from usnas2:
## -m is mountpoint. This command will try to restore file into same directory.
Another example of recovery:
uslgto2# recover -vvv -c usnas2 -s uslgto2.melonm.com No index was found for /nsr/scripts/restore/. The current working directory is /.
recover> cd "/vol/vol3_nas2/Common/SYSTEMS/Unix Systems/Documentation"
recover> relocate "/vol/vol3_nas2/Common/SYSTEMS/Unix Systems/Legato Operations/Restore"
recover> add WiltonHosts-Master-040309.xls /vol/vol3_nas2/Common/SYSTEMS/Unix
Systems/Documentation
1 file(s) marked for recovery recover> recover
Procedures to restore file index for expired saveset? • Locate SSID: mminfo -avot -c client_name
• Note: Check on the "level" of the target saveset that the user needs to recover, as if it is an incremental request to perform the instructions below for the most recent, along with all
incremental before the desired date.
• Obtain cloneid: mminfo -av -q ssid=XXXX -r cloneid
• Set the expiration date: nsrmm -e "mm/dd/yy" -S XXXX/YYYY
◦ The mm/dd/yy is the date the user would like the saveset to be
browsable from.
• Make saveset notrecyclable: nsrmm -o notrecyclable -S XXXX/YYYY
◦ Input "y" when it asks to mark the saveset as not recyclable. ◦ nsrmm -oy notrecyclable -S XXXX/YYYY
• Verify saveset status: mminfo -avot -c client_name
◦ From the output, the user will be able to see the saveset worked
on is now becomes "cr").
index:client_name
◦ Locate the index that is being backed up with the target saveset
and mark down its time.
• Start to recover index: nsrck -L7 -t "mm/dd/yy" client_name
◦ Input the time obtained from last step within " " ◦ eg, nsrck -L7 -t '03/28/09 19:46:53' usnas2
• Validate index recovered: mminfo -avot -c client_name
◦ From the output, the user will be able to see the saveset
worked??on? now becomes "cb".
• Repeat the above steps for any further saveset requires. If no more saveset operations are required, double check within NMC to make sure volumes required are available for recovery.
Storage
Key Processes
• The following 3 processes started from /etc/rc2.d/S95networker
◦ nsrexecd: client service. ◦ lgtolmd: license daemon
◦ nsrd: daemon to start the following 3 server processes (nsrmmd,
nsrindexd, nsrmmdbd)
nsrmmd: storage node daemon and write data to/from tape. nsrindexd: maintain & caching index of save sets.
nsrmmdbd: database daemon responsible for volume database under /nsr/mm/mmvolume6
◦ nsrexecd: main service running in client. Also running in servers
to backup servers themselves.
ssflags meaning: ssflags can contain following values: CvrSEiIF
C=continued v=valid
r=recoverable S=scanned-in
E=eligible for recycle i=incomplete
F=finished.
Additionally, N means "NDMP"
To validate clone set status, using "clflags". "a" or "E" means that it is corrupted cloneset.
• mminfo -avot -q "ssid=3973556614" -r "volume,ssid,ssflags,clflags"
Validating Legato hardware status • Shutdown networker server
• uslgto2# sjirdtag /dev/scsi/changer/c2t100000E00222AF71d0 • inquire -cl # see all tapes & changer
• sjirjc
• sjirjc 1.0.0 (where 1.0.0 is changer address. Locate it using inquire -cl).
• sjirdtag 1.0.0 (list details library information)
How to manually configure MSL6000 Jukebox using jbconfig? Download screen logs from here.
• inquire to get device path & serial number of drives; • sjisn to get element address with serial number;
• jbconfig to match the element with right device path using serial number.
• If there are storage node needs to be configured (2 nodes to see same drives), follow this instruction for configuration.
◦ The key to all same drive has 2 paths (one to Networker server
and another to Storage node).
• Each time hardware changed (eg, replace a drive), the library needs to be reconfigured in Legato using jbconfig.
NDMP Backup
• Regarding to incremental backup using NDMP
◦ The NetWorker software does not support incremental backup
settings for NDMP backups. If you select the incremental
setting, the NetWorker server performs a full backup. However, you can schedule level backups to function like incremental
backups. For example, complete a full backup on day 1 and level 1 backups on days 2 through 7.
"UPDATE=Y". This parameter actually instructs Ontap NDMP to update the
◦ dumpdates file which is used to determine what should be
backed up.
◦ To view all NDMP dump history & levels on NetApps, see
file: /vol/vol0/etc/dumpdates
• Backup Command: nsrndmp_save -c
hostname_of_nas_interface/client_name -s backupserver -I indexserver -M -T dump;
• NDMP Restore through Legato
◦ nsrndmp_recover -c usnas2 -s uslgto2 -m usorarestore:/u6 -S
3439474549 "/vol/vol2_nas2/WPSysMan/file.pdf" • NDMP Save through storage node public nic
◦ nsrndmp_save -c usnas2 -M -T dump -b "XING NDMP" -P
uslgto3v -y "11/07/08" /vol/vol0/home # uslgto3v is pub nic of storage node. Command is run in storage node.
How to exclude files/dirs from NDMP backup of NetApps filer?
To exclude files or directories from an NDMP backup of a NetApp filer, use the EXCLUDE= option in the NDMP Client resource's Application information field. EXCLUDE= takes a comma separated list of file and/or directory names to exclude from the backup. This is equivalent to the X option of the dump command. The following are rules for specifying names in the EXCLUDE= list:
• If a specific file or directory is to be excluded from the backup, the exact name of the file or directory must be used.
• An asterisk (*) can be used as the wildcard character. The
asterisk must be either the first or the last character of the string. Each string can have up to two asterisks (one in the front and one at the end).
• If a file or directory name contains a comma, the comma must be preceded by a backslash to denote special character. This is because comma is used in the EXCLUDE= string to separate values.
Here are some examples:
• To exclude the 'tmp' directory from a save set, use EXCLUDE=tmp
• To exclude all .PST files from the save set, use EXCLUDE=*.PST
• To exclude all files or directories that contain the characters 'old' in their names, use: EXCLUDE=*old*
• To exclude all of the above at the same time, use: EXCLUDE=tmp,*.PST,*old*
Recover Netapp filer's NDMP data
nwrecover: Before starting nwrecover GUI, please change the directory to
/filermountpoint, then start nwrecover with command "nwrecover -s (nwserver) -c (filername)"
or after starting nwrecover gui with command "nwrecover -s (nwserver) -c
(filername)" , you need to expand the filer's file in the Select:. For example, /vol/vol*.
Using command line: recover saveset
nsrndmp_recover -c filer_name -s (NWserver) -S ssid -m (filername)::target_dir
Recover single file: Recover
To recover NDMP savesession in FP, using uslgto3 as recover server due to NIC (pub&priv) issues.
Redirect "Bootstrap Completion" printout • In Configuration--> Notification, specify "cat >
/nsr/bootstrap/nsr_bootstrap_`/usr/bin/date '+20%y%m%d'`" as action of "Bootstrap Completion"
• Create directory /nsr/bootstrap and we will get updated bootstrap information each day.
• If possible, setup rdist so whole /nsr directory will be duplicated to remote server as backup.
Each day, the bootstrap information after each executed job will be saved into above directory instead of printing to physical
Some command line examples from this link . Local copy here. How to exclude a directory from scheduled backup using local directives?
• For Windows
◦ Create a file called nsr.dir under the parent directory (eg, C:\)
with content:
◦ skip: <name of dir>
◦ Make nsr.dir file hidden/readonly if necessary.
◦ Create a separate client with only this directory as saveset if it
needs to be backed up separately. • For Unix
◦ Create .nsr file under parent directories: ◦ << /olbackup1 >> ◦ +skip: . ◦ << /olbackup2 >> ◦ +skip: . ◦ << /olbackup3 >> ◦ +skip: . ◦ << /olbackup4 >> ◦ +skip: .
How to recover an expired saveset?
• Find the ssid(s) of the saveset: mminfo -av -q "client=myclient,savetime<mm/dd/yy"
• Mount the correponding tape: nsrjb -l -S <slot> -f <tape device> • Scan the tape for the ssid: scanner -i -S <myssid> -f
<filenumber> <tape device>
◦ Filenumber can be identified by: mminfo -q ssid -V
• Set the status of the saveset: nsrmm -S <myssid> -o notrecyclable
• (Optional) Specify a new browse or expiration time: nsrmm -S <myssid> -e mm/dd/yy -w mm/dd/yy
How to recover savesets on disk from another Legato server?
• Add disk folder into device;
• Mount disk devices • Recover
How to make a saveset nonrecyclable?
• Get the cloneid of saveid: mminfo -q client=uscoredb3 -r "ssid,cloneid" -t "-12 hours"
• nsrmm <-y> -o notrecyclable -S <saveid>/<cloneid> How to list files within ssid?
• mminfo -avot -c usuman0 -t "yesterday" -r "volume,savetime,nsavetime"
• nsrinfo -n all -t <nsavetime> usuman0
• Or using recover (slow): recover -n -S <ssid> • Or using GUI: nwrecover
How to remove expired saveset in the volume?
• Delete saveset from Media Index: nsrmm (-v) -d -S ssid/cloneid
◦ Above will not remove data from tape/disk, it only removes
index.
• To remove invalid saveset, clone the valid ones to other tape/adv_file devices and relabel them.
How to recycle tapes older than 1 month (keep in mind of saveset dependency)?
• mminfo -q "savetime > 1 month ago" -r "volume"
◦ Or: mminfo -av -q "savetime < 09/25/08" "XING.UnixD.0001"
• nsrmm -yo recyclable volume_name (loop through the list)
How to use nsrmm to change browse and retention policies? • Change the retention time to midnight, January 1, 2009. Change
the browse time to midnight, January 1, 2008.
◦ nsrmm -S 3315861249 -e "01/01/09 23:59:59" -w "01/01/08
23:59:59"
• Change the browse time to six months from the current date and time.
◦ nsrmm -S 5315861249 -w "6 months"
• Change the retention time to two years from the current date and time.
◦ nsrmm -S 3315861249 -e "2 years"
reclaim space? Details instruction download from here. • mminfo -m 1>volume 2>&1
• mminfo -r 'name, ssid, ssbrowse, ssretent, ssflags, state' -q volume=XING.UnixD.0001 1>query 2>&1
• Locate ssid
• nsrmm -d -S <ssid> to remove them from Index • nsrim -X --> run twice to reclaim the space
Script to remove "duplicate volumes" from media database by searching daemon.log. Download from here.
Tips
• Kick off starting group from command line: savegrp -l full -G Unix_Test
• Scan the content of tape: scanner -n /dev/rmt/6cbn (loaded tape into drive first)
• Save from client side: /usr/sbin/save -b "XING Unix" -l full /u4 • Disable VSS for Windows client: VSS:*=off (in "Save
Operations" of client definition)
• To enable legato automatic label blank tapes, choose "Auto Media Management" in the Jukebox's property.
• List VSS Writers in Windows 2003: vssadmin list writers • nsrmm (-C) --> display configured devices (tape/disk) and
mounted volumns
Retention Policy: There are 2 places to setup Retention policy: in the client level and in the pool level.
• The actual Retention policy will respect the longer one between client & media pool. The longer one will be the retention policy of savesets.
• Change Retention policy of saveset: nsrmm –S ssid –e “xx/xx/xx”
• To verify: mminfo -vp -q ssid=3528014006 How to control NSR Index size?
• Contribulting Factors: Number of clients; Number of files; Browsing Policy
◦ Retention policy has nothing to do with Index size, only
• Workaround:
◦ Clean up old decommissioned hosts index; ◦ Move the index to other location
How to completely remove client from Legato (eg, for reclaiming licenses)?
• Perform a full bootstrap backup for group which has clients to be removed (eg, savegrp -l full -O -G Unix_Prd). Record ssid in case media index needs to be restored.
• Delete the client in NMC • Stop nsr services
• Backup the client's index folder (/nsr/index/<client_name>) • Clear the cache by renaming \nsr\tmp\ folder to
\nsr\tmp_old_todaysdate • Start nsr services
• Temporarily set the Autostart function on all Groups to
'Disabled' (it is best to perform the following when NW is idle) • Run the following commands to verify, fix and purge the index
and mdb records:
◦ nsrck -L6 --> this will take a very LONG time. Please make sure
enough time is scheduled between backups.
◦ nsrim -X
• Set the Autostart function on groups that was changed in step 8 back to "Enabled".
How to remove old hostnames from registering in NSR server?
• Search old hostname in nsr indexes: /nsr/res/nsrdb & /nsr/res/nsrladb
◦ eg, grep */* /nsr/res/nsrdb & grep */* /nsr/res/nsrladb
• Edit the index file and remove the old hostname • Retry the backup
How to relocate client index to different location in Unix? Steps download from here.
How to configure library to unload tapes when idle?
Device->Library->Property->Timers(may need to enable diag mode in View)->Idle device timeout (set to 1, which is 1
minute).
• When tape drive is idle more than 1 minute, it will unload tapes back to slot.
How to increase number of cleans using cleaning tape?
Device->Library->Property->Default Cleanings (Change from 5 to 50)
# Cleaning tape can now be used 50 times instead of default 5 times.
# Using command line: nsrjb -U50 -S58 (#cleaning tape is in slot 58).
Windows
How to backup System State of Windows client? • Specify the following save sets:
• ASR:\
• SYSTEM DB:\ • SYSTEM FILES:\ • SYSTEM STATE:\
• In "Apps & Modules", specify "Save operations:" "VSS:*=off" How to backup VSS state of Windows client?
• C:\ • VSS ASR DISK:\ • VSS SYSTEM FILESET:\ • VSS OTHER:\ • VSS USER DATA:\ • VSS SYSTEM SERVICES:\ • VSS SYSTEM BOOT:\
How to recover Windows client using "System State" backup?
• Install the basic OS (install SP2 if original host has SP2)
• Rename the Host to original hostname and hardcode original IP address;
• Install NSR client after server renamed (that way the NSR client will identified itself with correct name), click all system state savesets and C: drive, then start "Direct Recovery".
• When done, removed hardcoded IP address and reboot the server. The server should be back with domain enabled. Legato Windows Agent Issues
• Not locate NSR server: First time when Legato Windows Agent starts, it will search for Networker server through broadcast. If Legato server is in a different network than agent, the search will fail and the agent gui will close.
◦ To fix this problem, append the server string after the
executable, eg: "C:\...winworkr.exe" -s <legato server>
• DMZ servers Name Resolution: Legato agent requires reverse-lookup working for both Networker server & storage node. For DMZ servers which have different DNS server, add
IP/Hostname of Legato servers into
C:\Windows\System32\drivers\etc\hosts
◦ If hosts file doesn't work, CREATE A NEW ONE. It is possibly
corrupted but not evident to you. Holy smokes, it works for me. Turning off vss for specific drive
• VSS:h:\=off
How to backup Exchange server?
• Backing up the information stores (msexch:IS) and recover them to the recovery storage group. Use the exchange task wizard to recover individual mailboxes from that.
• For the saveset specification, put MSEXCH: IS and MSEXCH: DS with a backup command of nsrxchsv.exe
NMDA
How to configure Lotus Notes backup using NMDA? • To keep configuration in MMC client property, configure:
◦ Saveset: NOTES
◦ Backup Command: nsrdasv -T Lotus
◦ Backup Config: copy and modify this file. Pay particular
attention to the followng to configurations.
NSR_CROSS_MOUNT_POINTS:"TRUE"; --> Important if mailboxes located in different file system
located in different file system
◦ Alias: Both long & short name of virtual & physical node
• To keep configuration in Lotus Notes client:
◦ Saveset: NOTES
◦ Backup Command: nsrdasv -z /nsr/apps/config/ nmda_lotus.cfg ◦ Alias: Both long & short name of virtual & physical node
How to restore Lotus Notes using NMDA through command line?
• Create nmda_lotus_recovery_cmd.cfg file, which will be used for restore.
◦ NSR_BACKUP_PATHS will specify objects needs to be
restored
◦ NSR_RELOCATION_DEST will specify path where data will
be restored to
◦ NSR_RECOVERY_TIME will specify which date data needs to
be restored
To get all information about backup objects, using following command and search output
nsrinfo -s <NWSRV> -n notes -X All <NOTESCLIENT>
• An issue was discovered through debugging that nsrnotesrc was trying to locate itself under " /usr/sbi/nsrnotesrc" instead of
"/usr/sbin/nsrnotesrc". It could be specific to environment or could be a program bug.
◦ nsrnotesrc: using `/usr/sbi/nsrnotesrc' for execpath, ◦ nsrnotesrc: /usr/sbi/nsrnotesrc started, pid 20705 ◦ nsrnotesrc: waitchild reaped, pid 20705
◦ nsrnotesrc: waitchild error on reap, pid 20705 (10)
◦ To workaround this issue, I created the directory "/usr/sbi" and
copied "nsrnotesrc" to it from "/usr/sbin". Restores were successful after this fix.
• This could be another environmental issue since
"/opt/ibm/lotus/notes/latest/linux/" was not in library path. The library "libxmlproc.so" is required by "libnotes.so" for restore.
◦ ln -sf /opt/ibm/lotus/notes/latest/linux/libxmlproc.so
/usr/lib/libxmlproc.so
which will be doing restore: export
LD_LIBRARY_PATH=/opt/ibm/lotus/notes/latest/linux
◦ Run ldd to validate: ldd
/opt/ibm/lotus/notes/latest/linux/libnotes.so
• As user doing the backup (eg, notes), run below command to recover mailbox into NSR_RELOCATION_DEST
◦ /usr/sbin/nsrnotesrc -z
/nsr/apps/config/nmda_lotus_recovery_cmd.cfg
How to restore Lotus Notes using NMDA through NMDA client GUI?
• Install NMDA for Windows GUI on Windows system which runs recovery
• Run "Directed Recovery"-->Choose Mail box-->"Recovery Options"-->Recovery
• Specify configuration file for recovery in the "Options". • The NMDA Gui will, by default, looking for a user called
"notes". If Lotus was running under different user name in the email server, create a dummy user, within the same group as other Notes users, in the email server.
◦ Make sure user "notes" has write permission to directory where
mailboxes are restored to (specify in "Recovery Options"). How to configure Exchange cluster backup using NME? -- NSR agent & NME into both nodes;
-- Create entry for each physical node in Networker server; -- Create entry for virtual node of Ex-cluster.
** For virtual node configuration:
-- Remote access list, specify SYSTEM@node1, SYSTEM@node2, Administrator@node1,
Administrator@node2
-- Backup Command in Apps/Modules: nsrxchsv -c usevs2 -- Specify Savesets. eg: MSEXCH:IS/SG1 or
MSEXCH:IS/SG1/Public Folder Store
How to configure standalone Exchange backup using NMM?
• Client:
◦ Backup command: nsrsnap_vss_save
◦ Save set: APPLICATIONS:\Microsoft Exchange 2007
• Application Information Variables:
◦ NSR_SNAP_TYPE=vss
◦ NSR_ALT_PATH=C:\mount_replica
• Group:
◦ Snapshot backup=yes (Just check the box) ◦ Snapshot Policy= Serverless Backup
◦ Snapshot Pool= Any configured pool for this backup
How to configure NMDA backup for DB2 (refer to DB2 section for more details)?
• Create configuration file for backup nmda_db2_backup.cfg • Run from command line: db2 backup db db2ins online load
"/usr/lib/libnsrdb2.so" options @/nsr/apps/config/nmda_db2.cfg How to restore data through NMDA backup (refer to DB2 section for more details)?
• Create configuration file for restore nmda_db2_restore.cfg • Run from command line: db2 restore db db2ins load
/usr/lib/libnsrdb2.so options
@nsr/apps/config/nmda_db2_restore.cfg
Daily Jobs
Tape Daily Recording & Archive
Made one process in place to archive daily tape list offsite and upload to Windows shared drive. 2 Points about this process: • We have daily clone in place at daytime, so any tapes within
past 24 hours will stay in libraries until clones complete.
• Legato server is in Unix and shared drive/storage for archive is in Windows, so we have to resorted to Samba/Expect script in another middleware system for uploading.
• Here is the actual process flow:
◦ Schedule this job (dailyTapes.sh) to generate 2 reporting files
(tapeReport.txt & tapeOffsite.txt). tapeReport.txt will be sending to various recipents by email while tapeOffsite.txt will be
uploaded to share drive to be (eventually) imported into SQL server media database.
◦ Above script will also copy 2 list files to Samba middleware
server. Using this job (nsrOfftapes.sh) at later time to upload both lists into share drive. The upload script will use Expect script (nsrOfftapes.exp) with Samba client to upload.
• Once above process is in place, we have an automated process to generate offsite tapes and the list will be archived in central server for future reference.
Daily Legato Routines (chkNSR.sh)
• checking interrupted overnight backup jobs
• Backup daily saveset Index (samething sending through email) to a local directory
• Replicate Index directory to a mirror site • Regular schedule change on jobs
• Clone some archive logs of Oracle databases
Daily Legato Media Database HealthCheck (nsrDBchk.sh) • Check media database status and archive result.
• "-d" option for debugging
• This script will need to run when NW server is idle (eg, no device mount/unmount). Otherwise, all device mount/unmount will be automatically cancelled and on pending.
How to calculate Data Domain usage per host through Networker (sizeCal.sh)
• Using "mminfo -avot" to get all savesets in DataDomain. Then produce a list of clients.
• Above script will calculate each host's space in DataDomain. • Note: Since size produced from mminfo didn't have floating
point, above arithmetic using BASH is OK. For adding
arithmetic numbers, you must use programming tool such as "bc".
◦ Here is a script using "bc" for arithmetic operations with
floating points (sizeFal.sh).
How to save daily bootstrap information?
you need to know which tapes contain it. You may be able to get some information from mminfo, but what happens if that client has been removed from Index (for reclaiming license, for
example).
• In Legato GUI-->Configuration-->Notification-->Bootstrap-->Action " cat > /nsr/bootstrap/nsr_bootstrap_`/usr/bin/date '+20%y%m%d'`"
• Now you should have a bootstrap text file saved to Legato directory each day every day.
How to customize Savegroup completion email?
By default, each day you will receive emails when savegroup completes. If you want to customize it (eg, add Savegroup name on the email subject), in the Notification-->Savegroup
completion, add name of script to Action field: /nsr/scripts/ savegroupEmail.sh
Copyright @ 2011 Melonm Documentation Project, All rights reserved.
