<Insert Picture Here>
Top 10 Things You Always Wanted to Know About Automatic Storage Management
But Were Afraid to Ask Nitin Vengurlekar
RAC/ASM Engineering Grid Product Strategy
<Insert Picture Here>
Agenda
•
ASM Overview – 2 minute tour
•
Top Ten Questions
• ASM Architecture
• ASM Configuration & Performance
• Backups
• ACFS
• Migration
• 3rd Party Software
ASM Architecture - Review
• Automatic Storage Management (ASM) instance
• Instance that manages the diskgroup metadata
• Disk Groups
• Logical grouping of disks
• Determines file mirroring options
• ASM Disks
• LUNs presented to ASM
• ASM Files
• Files that are stored in ASM disk groups are called ASM files, this includes database files
ASM Architecture
A Brief Overview Pool of Storage ASM Instance Server Non–RAC Database Oracle DB InstanceASM Process Architecture
A Brief Overview Clustered Pool of Storage ASM Instance ASM Instance ASM Instance ASM Instance Clustered Servers RAC or Non–RAC Databases Oracle DB Instance Oracle DB Instance Oracle DB Instance Oracle DB Instance Database ASM Instance Oracle DB Instance Database Database<Insert Picture Here>
Top 10 ASM Questions
ASM Architecture
Top 10 ASM Questions
Q. What init.ora parameters does a user need to configure for ASM instances?
A. The default parameter settings work perfectly for
ASM. The only parameters needed for 11g ASM:
• PROCESSES
• ASM_DISKSTRING*
• ASM_DISKGROUPS*
Top 10 ASM Questions
Q. How does the database interact with the ASM instance and how do I make ASM go faster?
A. ASM is not in the I/O path so ASM does not impede
the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.
RDBMS and ASM Instance Interaction
Server
Operating System
DATABASE ASM
(1) Database opens file
(1A) OPEN
(1B) Extent Map
(2) Database Reads file
(2 A ) R E A D (2 B ) I /O C o m p le te s
(3) Database Creates file
(3A) CREATE (3B ) A llo c a te s f ile (3C) Extent Map (3 D ) I n it ia liz e s f ile (3D) Commit
Top 10 ASM Questions
Q. Do I need to define the RDBMS
FILESYSTEMIO_OPTIONS parameter when I use ASM?
A. No. The RDBMS does I/O directly to the raw disk
devices, the FILESYSTEMIO_OPTIONS parameter is only for filesystems.
Top 10 ASM Questions
Q. Don’t I lose all the advanced filesystem features when I move to ASM; e.g., directIO, write coalescing, and pre-fetch?
A. Yes, but that’s okay.
• Most of the filesystem features mentioned, though good for
general file data performance, interfere and fractionalize the benefits inherently provided by the database; e.g.,
• DBWR & LGWR does write coalescing, and user processes do sequential pre-fetches
Top 10 ASM Questions
Q. This is cool that ASM can now store Vote and OCR files. But how does CSS and CRS startup in this configuration?
A. It just does ☺
• There are two keys processes and a lot of crafty coding to
Top 10 ASM Questions
ASMCMD> pwd
+DATA/rst-cluster/OCRFILE ASMCMD> ls -l
Type Redund Striped Time Sys Name
OCRFILE UNPROT COARSE JUN 25 11:00:00 Y REGISTRY.255.718984285 ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB
Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 203824 193028
CSS-CRS-ASM Instance Architecture
ASM
GPnP
1. Get ASM disk string
2. Do ASM disk discovery to locate vote disk
ASM Disk CRS ASM Disk DATA DG SPFILE Vote OCR
3. Once Vote disk located, startup CSS
OHASD
4. ASM mounts DATA dg
Agents starts GPnP, CSS and ASM
CSS
5. CRS open OCR in well known ASM dg location
<Insert Picture Here>
Top 10 ASM Questions
ASM Configuration
Top 10 ASM Questions
Q. Do I need 11gR2 Grid Infrastructure to use ASM
A. Yes. ASM is now part of Grid Infrastructure, which
includes, Clusterware, ASM and ACFS. So you’ll to Install GI to use ASM
• In 11gR2 there are two options for install – GI for Standalone Server (aka Oracle Restart) and GI for Clusterware
Top 10 ASM Questions
Q. We have a 16 TB database. I’m curious about the number of disk groups we should use; e.g. 1 large disk group, a couple of disk groups, or otherwise? What about a database consolidation scenario
A. For VLDBs you will probably end up with different
storage tiers; e.g with some of our large customers
they have Tier1 (RAID10 FC), Tier2 (RAID5 FC), Tier3 (SATA), etc. Each one of these is mapped to a
Top 10 ASM Questions
Q. What is the best LUN size for ASM
A. There is no best size! In most cases the storage
team will dictate based on their standardized LUN
size. The ASM admin merely has to communicate the ASM Best Practices and application characteristics to storage folks :
• Need equally sized / performance LUNs
• Minimum of 4 LUNs
• The capacity requirement
• The workload characteristic (random r/w, sequential r/w) & any response time SLA
Using this info, and their standards, the storage folks should build a nice LUN group set for you.
Top 10 ASM Questions
Q. In 11gR2 can my RDBMS and ASM instances run different versions?
A. Yes. But since ASM is now part of GI stack, it must be
at the highest version. Keep in mind, there’s two components of compatibility:
• Software compatibility
• Diskgroup compatibility attributes:
• compatible.asm
Disk Group Compatibility Example
• Need to have compatible.asm set to 11.2.0.1 for
OCR/Vote files and SPFILE in ASM
• ACFS also needs 11.2.0.1 compatible.asm
• Advance compatible.asm
• ALTER DISKGROUP data SET ATTRIBUTE
Top 10 ASM Questions
Q. Where do I run my database listener from; i.e., ASM HOME or DB HOME?
A.
• For 11gR2, the SCAN listener is run from GI Home,
and database listener from DB HOME.
• For pre-11gR2, it is recommended to run the listener
from the ASM HOME. This is particularly important for RAC env, since the listener is a node-level resource. In this config, you can create additional [user] listeners from the database homes as needed.
<Insert Picture Here>
Top 10 ASM Questions
Backups
Top 10 ASM Questions
Q. How do I backup my ASM instance?
Top 10 ASM Questions
Q. When should I use RMAN and when should I use ASMCMD copy?
A. RMAN is the recommended and most complete and
flexible method to backup and transport database files in ASM.
ASMCMD copy is good for copying single files
• Supports all Oracle file types
• In some cases. can be used to instantiate a Data Guard environment
• Does not update the controlfile
ASMCMD Copy
ASMCMD> ls
+fra/dumpsets/expdp_5_5.dat
ASMCMD> cp expdp_5_5.dat [email protected]:+DATA/dumpsets/ex pdp_5_5.dat source +fra/dumpsets/expdp_5_5.dat target +DATA/dumpsets/expdp_5_5.dat copying file(s)... file, +DATA/dumpsets/expdp_5_5.dat, copy committed.
<Insert Picture Here>
Top 10 ASM Questions
Migration
Top 10 ASM Questions
Q. We are migrating to a new storage array. How do I move my ASM database from storage A to storage B?
A.
• Given that the new and old storage are both visible to
ASM, simply add the new disks to the ASM disk group and drop the old disks. ASM rebalance will migrate
data online.
• For pre-11gR2, See Note 428681.1, which covers how
Top 10 ASM Questions
ASM_SQL> alter diskgroup DATA drop disk data_legacy1, data_legacy2, data_legacy3 add disk ‘/dev/sddb1’, ‘/dev/sddc1’, ‘/dev/sddd1’;
ASM Rebalancing
• Automatic online rebalance whenever storage configuration changes
• Only move data proportional to storage added
• No need for manual I/O tuning
ASM Rebalancing
• Automatic online rebalance whenever storage configuration changes
• Online migration to new storage
ASM Rebalancing
• Automatic online rebalance whenever storage configuration changes
• Online migration to new storage
ASM Rebalancing
• Automatic online rebalance whenever storage configuration changes
• Online migration to new storage
ASM Rebalancing
• Automatic online rebalance whenever storage configuration changes
• Online migration to new storage
Top 10 ASM Questions
Q. Is it possible to unplug an ASM disk group from one platform and plug into a server on another platform (for example, from Solaris to Linux)?
A. No. Cross-platform disk group migration not
supported. To move datafiles between endian-ness platforms, you need to use XTTS, Datapump or
<Insert Picture Here>
Top 10 ASM Questions
ACFS
46
3rd Party FS
Application
Extending ASM to Manage ALL Data
Automatic Storage Management (ASM)
ASM Instance Managing Oracle DB Files
ASM Cluster & Single Node File System
(ACFS)
Database RAC
ACFS Snapshot
ASM Files
Dynamic Volume Manager
Dynamic Volumes ACFS FS Oracle Bin 3 rdParty FS DB Datafiles DB Datafiles DB Datafiles DB Datafiles DB Datafiles DB Datafiles
OCR & Voting files
OCR & Voting files
ASM Disk Group
47
• General purpose scalable file system
• Journaling, extent based
• Single node and cluster
• POSIX, X/OPEN file system solution for UNIX/Linux
• Windows file system solution for Windows platforms
• Accessible through NAS protocols (NFS, CIFS)
• Leverages ASM technology
• Integrated with Oracle Clusterware for cluster support
• Multi OS platform (Linux and Windows at initial release)
• Integrated with Oracle system mgt tools
• Oracle installation and configuration
• Enterprise Manager and ASM Storage mgt tools
• Native OS File System Management tools
What is ASM Cluster File System
(ACFS)
48
ACFS Features
• Provides filesystem snapshots (FCOW)
• File system integrity and fast recovery via ACFS metadata checksums and journaling.
• ACFS designed as a peer to peer, multi-node, shared file system model and delivers coherent data access
• ACFS file system is installed as a dynamically loadable OS VFS driver
• Starting with RHEL5, Redhat now supports a 'white list'
-kernel APIs which they commit they will not change in updates or patches. APIs used by ACFS-ADVM were added to their 'white list'.
• Customers should be able to install an update or patch to the kernel and our drivers should not be impacted
Top 10 ASM Questions
Q. Is ACFS supported on other platforms besides Linux
Top 10 ASM Questions
Q. Can ACFS be used to store database datafiles? What about archive logs?
A. No. Currently we will not support database file to be
stored in ACFS. This is due to performance reasons. Though you can do this in test/Q&A environments
Top 10 ASM Questions
Q. Can I use ACFS to store BFILE data or other non-database related data
A. Yes. ACFS is POSIX compliant filesystem, and thus
Top 10 ASM Questions
Q. Will ACFS support other Data services, such advanced cloning, replication, de-dupe, etc..
A. Yes. ACFS Replication will be introduced in the next
patchset release. Other advanced features are part of the roadmap.
<Insert Picture Here>
Top 10 ASM Questions
3
rdParty Software
Top 10 ASM Questions
Q. How does ASM work with multipathing software?
A. It works great! Multipathing software is at a layer
lower than ASM, and thus is transparent.
You may need to adjust ASM_DISKSTRING to specify only the path to the multipathing pseudo devices.
ASM/RDBMS /dev/<pseudo> Multipath driver /dev/sda1 /dev/sdb1 Controller Controller Cache Disk IO cloud
Top 10 ASM Questions
Q. Is ASM constantly rebalancing to manage “hot spots”?
I/O Distribution
• ASM spreads file extents evenly across all disks in disk group • Since ASM distributes extents evenly, there are no hot spots
Average IOPS per disk during OLTP w orkload
0 50 100 150 200 250 300 Total Disk IO P S FG1: - cciss/c0d2 FG1: - cciss/c0d3 FG1: - cciss/c0d4 FG1: - cciss/c0d5 FG1: - cciss/c0d6 FG2: - cciss/c0d2 FG2: - cciss/c0d3 FG2: - cciss/c0d4 FG2: - cciss/c0d5 FG2: - cciss/c0d6 FG3: - cciss/c0d2 FG3: - cciss/c0d3 FG3: - cciss/c0d4 FG3: - cciss/c0d5 FG3: - cciss/c0d6 FG4: - cciss/c0d2 FG4: - cciss/c0d3 FG4: - cciss/c0d4 FG4: - cciss/c0d5 FG4: - cciss/c0d6
Summary:
• ASM requires very few parameters to run
• ASM based databases inherently leverage raw disk
performance
• No additional database parameters needed to support
ASM
• Mixed ASM-database version support
• Facilitates online storage changes
• RMAN recommended for backing up ASM based
databases
• Spreads I/O evenly across all disks to maximize
ASM Collateral and Content
http://www.oracle.com/technology/asm
•
ASM 11g New Features
•
ASM Best Practices
•
ASM vendor papers
<Insert Picture Here>
Top 10 ASM Questions
Extra credit questions
Top 10 ASM Questions
Q. Is ASMLIB required on Linux systems and are there any benefits to using it?
A. ASMLIB is not required to run ASM, but it is certainly
recommended.
ASMLIB has following benefits:
• Simplified disk discovery
• Persistent disk names
Top 10 ASM Questions
Q. Is it possible to do rolling upgrades on ASMLIB in a RAC configuration
A. ASMLIB is independent of Oracle Clusterware and
Oracle Database, and thus can be upgraded on its own
