Hosted by
Keys to Optimizing Your
Backup Environment:
Tivoli Storage Manager
John Merryman
GlassHouse Technologies, Inc.
Hosted by
Introduction
z
Audience Profile
z
Storage Management Interdependence
z
Backup Pain Points
z
TSM Architecture
•
Server
•
Storage
•
Network
•
Clients
Hosted by
Audience Response
z
Audience Profile
z
Your TSM Backup/Archive Environment
z
Number of Years using TSM
z
Number of TSM Administrators
Storage Management Interdependence
z
Storage and Production Backup
•
Growing at 65% CAGR (Gartner)
•
Consuming 48% of IT hardware budgets (Gartner)
z
Disaster Recovery
•
Enabling Recovery – RPO/RTO impact
z
Archiving
•
Compliance
•
Storage efficiency
•
Disposal of data
•
Application Performance
“It’s no longer just
backup, it’s data
management.”
Hosted by
TSM Pain Points
Business Problems
Backup reliability
Overwhelming data growth
Database and email data
explosion
Limited budgets and staff
Regulatory / data retention
requirements
Lack of data management
policies
Poor operational practices
Technology Problems
TSM architecture scalability
Lack of TSM / OS tuning
Network architecture
Client-side issues (OS, antivirus,
network)
Increasingly complex requirements
Shrinking backup windows
Disconnect between application
design and storage management
Solution
Optimize and right-size existing
infrastructure
Improve processes and
procedures
Formalize TSM program
management
Benefits
Lower TCO for TSM environment
Proactive management model
Lowered risks and exposures
Scalable and manageable
environment
Hosted by
Business Problems
Backup reliability
Overwhelming data growth
Database and email data explosion
Limited budgets and staff
Regulatory / data retention requirements
Lack of data management policies
Poor operational practices
Hosted by
Technology Problems
•
TSM architecture scalability
•
Lack of TSM / OS tuning
•
Network architecture
•
Client-side issues (OS, antivirus, network)
•
Increasingly complex requirements
•
Shrinking backup windows
•
Disconnect between application design and storage
management
Hosted by
Solution
•
Optimize and right-size existing infrastructure
•
Improve processes and procedures
•
Formalize TSM program management
Hosted by
Benefits
•
Lower TCO for TSM environment
•
Proactive management model
•
Lowered risks and exposures
•
Scalable and manageable environment
TSM Pain Points
TSM Client Administration
7%
61%
7%
3%
3%
2%
2%
3%
2%
2%
8%
TSM Certifications,
De-Certifications, Re-Certifications
TSM Client Problem Resolution
TSM Client Upgrades / Change
Management
TSM Client Process Improvement
Testing and Certifying New Client
Technologies
Performing Restores
TSM Server / Device Problem
Resolution
TSM Server Administration
TSM Server Monitoring, Reporting,
Planning
Testing and Certifying New Server
Technologies
Joint Effort Problem Resolution for
TSM Client Issues
Hosted by
TSM Architecture Overview
LAN
Database
Recovery Log
TSM
Server
SAN
TSM Clients
Hosted by
TSM
Clients
SD HEWLETT PACKARD SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch P OWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLED
TSM
Server
Tape/SAN/Disk
Technology
Network
Network
SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch P OWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLED S D HEWLETT PACKARD
Tape/SAN/Disk
Technology
TSM
Clients
TSM
Server
TSM Architecture
Hosted by
TSM Server Performance
z
Hardware
•
CPU, Memory, HBAs
Server backplane/bus architecture
HBA location
z
Operating System Tuning
•
VMTUNE for AIX has a significant impact
Must be tuned specifically for environment
•
Asynchronous I/O (requires TSM settings too)
•
Direct I/O (requires TSM settings too)
•
Network options tuning
Hosted by
TSM Application Tuning
z
TSM Application Tuning
•
Database Buffer Pool Sizing
Server Options bufpoolsize parameter
(up to 50%)
Selftunebufpoolsize parameter (up to 10%)
Relates closely to page space sizing
•
Mirrorwrite db parallel and DBpageshadow
•
Txngroupmax, movebatchsize, movesizethresh
Hosted by
TSM Application Tuning, II
z
64-bit and 32-bit code levels (and OS levels)
z
TSM performance tuning guide
•
http://publib.boulder.ibm.com/tividd/td/TS
MM/SC32-9101-01/en_US/HTML/SC32-9101-01.htm
z
Maintain a link between operating system,
application, and device performance analysis.
TSM Server Performance
z
Other Considerations
•
Backup versioning policies have an effect.
•
Schedule randomization versus static schedules
•
Spread production workload across backup window.
•
Control batch TSM processes to avoid overlap for
resource intensive processes.
•
Low-level policies can have a high-level impact.
shrstatic versus shrdynamic
Changing retries settings
Hosted by
TSM Server Operations
Hosted by
Ideal TSM Batch Processing
0:
00
1:
00
2:
00
3:
00
4:
00
5:
00
6:
00
7:
00
8:
00
9:
00
10
:00
11
:00
12
:00
13
:00
14
:00
15
:00
16
:00
17
:00
18
:00
19
:00
20
:00
21
:00
22
:00
23
:
00
Backup STG Process 360
Backup STG Process 399
Backup STG Process 400
Database Backup Process 472
Migration Process 365
Migration Process 473
Migration Process 474
Expire Inventory Process 479
Space Reclamation Process 480
Space Reclamation Process 481
Space Reclamation Process 482
Space Reclamation Process 483
Hosted by
Poorly Optimized TSM Batch Processing
11:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:0022:0023:000:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:0022:0023:000:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:0022:0023:000:001:002:003:004:005:006:007:008:009:00
Backu p STG Proc ess 3 60 Migra tion Proce ss 36 5 Migra tion Proce ss 36 6 Migra tion Proce ss 36 7 Migra tion Proce ss 36 8 Migra tion Proce ss 36 9 Migra tion Proce ss 37 0 Migra tion Proce ss 37 1 Migra tion Proce ss 37 2 Migra tion Proce ss 37 3 Migra tion Proce ss 37 4 Migra tion Proce ss 37 5 Backu p STG Proc ess 3 76 Backu p STG Proc ess 3 77 Backu p STG Proc ess 3 78 Backu p STG Proc ess 3 79 Migra tion Proce ss 38 0 Migra tion Proce ss 38 1 Migra tion Proce ss 38 2 Migra tion Proce ss 38 3 Migra tion Proce ss 38 4 Migra tion Proce ss 38 5 Migra tion Proce ss 38 6 Migra tion Proce ss 38 7 Migra tion Proce ss 38 8 Migra tion Proce ss 38 9 Migra tion Proce ss 39 0 Migra tion Proce ss 39 1 Migra tion Proce ss 39 2 Migra tion Proce ss 39 3 Migra tion Proce ss 39 4 Migra tion Proce ss 39 5 Migra tion Proce ss 39 6 Migra tion Proce ss 39 7 Backu p STG Proc ess 3 98 Backu p STG Proc ess 3 99 Backu p STG Proc ess 4 00 Backu p STG Proc ess 4 01 Backu p STG Proc ess 4 02 Backu p STG Proc ess 4 03 Backu p STG Proc ess 4 04 Backu p STG Proc ess 4 05 Migra tion Proce ss 40 6 Migra tion Proce ss 40 7 Migra tion Proce ss 40 8 Migra tion Proce ss 40 9 Migra tion Proce ss 41 0 Migra tion Proce ss 41 1 Migra tion Proce ss 41 2 Migra tion Proce ss 41 3 Backu p STG Proc ess 4 14 Backu p STG Proc ess 4 15 Backu p STG Proc ess 4 16 Backu p STG Proc ess 4 17 Migra tion Proce ss 41 8 Migra tion Proce ss 41 9 Migra tion Proce ss 42 0 Migra tion Proce ss 42 1 Migra tion Proce ss 42 2 Migra tion Proce ss 42 3 Migra tion Proce ss 42 4 Migra tion Proce ss 42 5 Migra tion Proce ss 42 6 Migra tion Proce ss 42 7 Migra tion Proce ss 42 8 Migra tion Proce ss 42 9 Migra tion Proce ss 43 0 Migra tion Proce ss 43 1 Migra tion Proce ss 43 2 Migra tion Proce ss 43 3 Migra tion Proce ss 43 4 Migra tion Proce ss 43 5 Migra tion Proce ss 43 6 Migra tion Proce ss 43 7 Migra tion Proce ss 43 8 Migra tion Proce ss 43 9 Migra tion Proce ss 44 0 Migra tion Proce ss 44 1 Migra tion Proce ss 44 2 Migra tion Proce ss 44 3 Migra tion Proce ss 44 4 Migra tion Proce ss 44 5 Migra tion Proce ss 44 6 Datab ase B ackup Proc ess 4 47 Migra tion Proce ss 44 8 Migra tion Proce ss 44 9 Migra tion Proce ss 45 0 Migra tion Proce ss 45 1 Migra tion Proce ss 45 2 Migra tion Proce ss 45 3 Migra tion Proce ss 45 4 Migra tion Proce ss 45 5 Migra tion Proce ss 45 6 Migra tion Proce ss 45 7 Migra tion Proce ss 45 8 Migra tion Proce ss 45 9 Backu p STG Proc ess 4 60 Backu p STG Proc ess 4 61 Backu p STG Proc ess 4 62 Backu p STG Proc ess 4 63 Backu p STG Proc ess 4 64 Backu p STG Proc ess 4 65 Backu p STG Proc ess 4 66 Backu p STG Proc ess 4 67 Backu p STG Proc ess 4 68 Backu p STG Proc ess 4 69 Backu p STG Proc ess 4 70 Backu p STG Proc ess 4 71 Datab ase B ackup Proc ess 4 72 Migra tion Proce ss 47 3 Migra tion Proce ss 47 4 Migra tion Proce ss 47 5 Migra tion Proce ss 47 6 Migra tion Proce ss 47 7 Migra tion Proce ss 47 8 Expir e Inv entor y Pro cess 479 Space Recl amati on Pr ocess 480 Space Recl amati on Pr ocess 481 Space Recl amati on Pr ocess 482 Space Recl amati on Pr ocess 483 Space Recl amati on Pr ocess 484 Space Recl amati on Pr ocess 485 Space Recl amati on Pr ocess 486 Space Recl amati on Pr ocess 487 Datab ase B ackup Proc ess 4 88 Space Recl amati on Pr ocess 489 Space Recl amati on Pr ocess 490 Space Recl amati on Pr ocess 491 Space Recl amati on Pr ocess 492 Space Recl amati on Pr ocess 493 Space Recl amati on Pr ocess 494 Space Recl amati on Pr ocess 495 Space Recl amati on Pr ocess 496 Space Recl amati on Pr ocess 497 Space Recl amati on Pr ocess 498 Space Recl amati on Pr ocess 499 Datab ase B ackup Proc ess 5 00 Migra tion Proce ss 50 1 Migra tion Proce ss 50 2 Migra tion Proce ss 50 3 Migra tion Proce ss 50 4 Migra tion Proce ss 50 5 Migra tion Proce ss 50 6 Space Recl amati on Pr ocess 507 Space Recl amati on Pr ocess 508 Datab ase B ackup Proc ess 5 09 Space Recl amati on Pr ocess 510 Space Recl amati on Pr ocess 511 Datab ase B ackup Proc ess 5 12 Migra tion Proce ss 51 3 Migra tion Proce ss 51 4 Migra tion Proce ss 51 5 Migra tion Proce ss 51 6 Migra tion Proce ss 51 7 Migra tion Proce ss 51 8 Datab ase B ackup Proc ess 5 19 Space Recl amati on Pr ocess 520 Migra tion Proce ss 52 1 Migra tion Proce ss 52 2 Migra tion Proce ss 52 3 Migra tion Proce ss 52 4 Migra tion Proce ss 52 5 Migra tion Proce ss 52 6 Datab ase B ackup Proc ess 5 27 Backu p STG Proc ess 5 28 Backu p STG Proc ess 5 29 Backu p STG Proc ess 5 30 Backu p STG Proc ess 5 31 Backu p STG Proc ess 5 32 Backu p STG Proc ess 5 33 Backu p STG Proc ess 5 34 Backu p STG Proc ess 5 35 Migra tion Proce ss 53 6 Migra tion Proce ss 53 7 Migra tion Proce ss 53 8 Migra tion Proce ss 53 9 Migra tion Proce ss 54 0 Backu p STG Proc ess 5 41 Backu p STG Proc ess 5 42 Backu p STG Proc ess 5 43 Backu p STG Proc ess 5 44
10/ 11/20 02 10/1 2/200 3 10/1 3/200 3 10/1 4/200 3
Hosted by
SD HEWLETT PACKARD SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLED
TSM
Server
Tape/SAN/Disk
Technology
Network
TSM
Clients
Tape/SAN/Disk
Technology
S D HEWLETT PACKARDNetwork
TSM
Clients
TSM
Server
SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLEDTSM Architecture
Hosted by
SAN Architecture
z
Disk Fabric Design
•
Dedicated SAN/Disk resources for Database/Logs
•
Dedicated SAN/Disk for storage pools
z
Tape Fabric Design
•
Load Balancing and design is important
•
Develop Ratios based on throughput and SAN fabric
1 GB Fabric w/ 3590-E1A Drives
•
1:3 Ratio HBA: Drive
2 GB Fabric w/ 3590-E1A Drives
Disk Architecture
z
Disk architecture
•
Fast, redundant, flexible and scalable
z
File system design
•
Create naming conventions and plan for scalability.
•
Raw volumes versus file systems
•
Create file system schema with disk layout in mind.
•
Database/log performance (Random I/O)
•
Storage volume performance (sequential)
Hosted by
Disk Architecture, II
z
Storage device architecture
•
Design/layout is key for sequential versus random
disk performance requirements
•
TSM is often the most I/O intensive application in
the datacenter.
•
Disk performance tuning is often necessary.
Hosted by
Disk Storage Pools
z
Capacity Planning for disk storage pools
z
Plan for using Migration Delay for critical client
data.
•
A note on storage pool caching and database
performance
z
Disk is traditionally used in TSM as a staging area
for daily migration to tape.
Hosted by
Disk as a Primary Storage Pool
z
Considerations
•
Detailed Capacity planning methods for disk storage
are required.
•
TSM Backup Policies have to be carefully planned.
•
Formatting and Recovering Terabytes of *.dsm files
is incredibly time and resource intensive.
•
Performance-oriented disk storage pool design
becomes even more important.
•
Copies to tape will still be required, and
simultaneous R/W can be a big issue for S-ATA
technologies.
Hosted by
Tape Architecture
z
Size and Type of Library
•
Robot speed, Compression, Scalability, etc.
z
Number and Type of Tape Drives
•
R/W Speed, Caching, Mount/Seek Times, etc.
z
Media Formats
•
Create a media migration strategy and plan.
z
Consider emerging tape technologies.
•
Tape is not dead and is not dying.
•
High-end tape I/O is faster than low-end disk I/O.
•
The real issue is random vs. sequential access to
data.
Hosted by
Tape Storage Pools
z
Stream large database objects data directly to tape.
•
Lowers disk pool requirements
•
Lowers TSM batch processing time
•
Optimizes use of tape resources during backup window
•
Schedule and resource planning is key.
•
Resource Utilization and Maxnummp settings
z
Synchronous Copy Storage Pool Writes (“”)
•
Can reduce batch window significantly
•
Use only for large object backup clients.
z
Recovery optimized TSM tape storage pools
SD HEWLETT PACKARD SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TY
LINK O KDISABLED
TSM
Server
Tape/SAN/Disk
Technology
Network
TSM
Clients
SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLEDS D HEWLETT PACKARD
TSM
Server
Tape/SAN/Disk
Technology
TSM
Clients
TSM Architecture
Network
Hosted by
Network Overview
z
Network Architecture
•
Private / Public network configurations
z
Type of Architecture
•
100 Mb and Gigabit IP architectures are the defacto standard.
z
Develop Network Architecture Standards.
•
Map network standards to SLAs and to service delivery.
•
Publish metrics on network throughput.
•
Enforce Standards.
z
Advancing a network architecture can be cost prohibitive
and time consuming…so can poor recovery times and data
loss.
Hosted by
Network Design
z
Dedicated Private VLAN architectures
•
Logical network segregation and physical network
segregation
•
Use jumbo frames and gigabit VLANs.
Improves streaming efficiencies to tape
9000 and 9004 MTU size differences
DNS Considerations
z
Multiple HBA’s for client network load balancing
•
Etherchannel configurations
•
Load balancing for network, switches, server HBAs
•
Don’t use auto-negotiate network card settings.
Hosted by
SD HEWLETT PACKARD SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLED
TSM
Server
Tape/SAN/Disk
Technology
Network
TSM
Clients
TSM
Clients
SD 4567891011 10/ 100 BASE TX 1 2 3 5411 Enterprise Switch POWER MGMT 45 6 78 9 1 0 1111223333 3R R 10 00 BASE X AMBER = GREEN = FL ASHING GREEN =AC TIVI TYLINK O KDISABLEDTSM
Server
Tape/SAN/Disk
Technology
Network
TSM Architecture
S D HEWLETT PACKARDHosted by
TSM Client Performance
z
Fact: Client machines are often sized for
application workload, not backup workload.
z
Client Tuning Parameters
•
Resource Utilization (2-10)
•
Memoryefficientbackup
•
Virtualmountpoint
z
Network / Transaction Tuning Parameters
•
Tcpnodelay
•
Tcpwindowsize
•
Tcpbufsize
•
Txnbytelimit
Hosted by
TSM Client Configuration
z
File System Design and Type
•
File system size and number impacts memory
consumption.
•
TSM client can be memory intensive (300 MB
Memory / Million Objects).
z
Include/Exclude list size has a direct impact on
processing overhead.
TSM Client Configuration, II
z
Journal-based backups
•
Data drives only
•
Journal size is limited to 2 GB
z
Image backups
z
LAN-free backups
Hosted by
TSM Client Performance
Backup Object Success Rates
Chart 2
1
10
100
1,000
10,000
100,000
1,000,000
INAA2
06
8
INAA2
07
3
INDA0
00
1
INDA0
00
3
INDA2
00
0
INDA2
00
1
INDA2
01
0
INDA2
01
4
IN
EA
20
00
IN
EA
20
07
IN
EA
20
08
IN
EA
20
15
IN
EA
20
21
IN
P
A
20
00
IN
PSW
B
01
IN
S
A
20
05
IN
S
A
20
06
Node Name
Av
er
ag
e Nu
m
b
e
r o
f
O
b
jec
ts
Average Objects Inspected
Average Objects Backed Up
Average Objects Failed
Hosted by
TSM Client Performance
z
Additional Considerations
•
Data characteristics
Data type, volume, volatility
Number of files
•
Client workloads and backup schedule
•
Client Software Version
•
Network load and utilization
•
Communication with application architects
and developers is increasingly important.
SD
HEWLETT PACKARD
Hosted by
TSM Client Performance
Client Versions
Ve rsion 4 Re lease 1 Level 2.0, 45 Clients
V ersion 5 Release 1 Level 5.0, 27 Clients
Version 4 Release 1 Level 2.12, 24 Cli ents V ersion 3 Release 1 Level 0.5, 21 Clients
(?), 14 Clien ts Version 5 Release 1 Level 5.14, 12 Clients
Version 4 Release 1 Level 1 .0, 11 Clients Version 4 Release 2 Level 1.0, 8 Clients
Version 4 Release 2 Level 0.0, 4 Clie nts Version 3 Release 1 Level 0.7, 4 Clients
Version 3 Release 1 L evel 0.1, 3 Cli ents Version 3 Release 1 Level 0.3, 3 Clients
Version 4 Re lease 1 Level 0.0, 2 Clients Version 5 Release 1 Level 6.0, 1 Clie nts
Version 3 Rele ase 1 Level 0.8, 1 Clients Ve rsion 3 Release 7 Level 1.0, 1 Clients