NEC Storage NV Series

(1)

NEC Storage NV Series

Guide to Oracle Storage Compatibility Program

Network Attached File Server

Snapshot Technologies

iS-WP-04-001

Rev-1.00 February, 2005

(2)

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT ARE DISCLAIMED.

NEC Storage NV Series and NEC Storage Manager/IP are trademarks or registered trademarks of NEC Corporation in Japan.

Oracle, Oracle9i and other product names of Oracle are trademarks, registered trademarks of Oracle Corporation in the U.S. and other countries.

Sun, Sun Microsystems, Sun logos, and Solaris are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries.

Microsoft and Windows are registered trademarks of Microsoft Corporation in the U.S. and other countries.

All other brands or products are trademarks or registered trademarks of their respective holders are should be treated as such.

(3)

1. Introduction

Recently, as the demand for such tasks as data sharing in multi-OS environments and cost cuts in storage management has risen, the use of NAS (Network Attached Storage), the storage appliance product, has increased. Moreover, as the Internet moves to broadband and the number of services delivering high-volume content grows, the demand for a large-capacity, efficient, high-end NAS product is also expanding.

NEC Storage* NV Series is a high-end NAS product providing high performance, high availability and high reliability and adopts the latest architecture features, such as a high performance TOE (TCP/IP Offload Engine), an OS specifically optimized for NAS, large capacity non-volatile memory (NVRAM), a remote replication capability, Snapshot function and more.

Based on the Oracle Storage Compatibility Program (OSCP), Oracle provides snapshot tests guideline to verity a file server to be used for Oracle database. This document describes how to create snapshots for backup and how to use the snapshots for recovery for the Oracle database by using the OSCP test kit provided Oracle Corporation and the snapshot function of the NEC Storage NV Series, and also shows the verification test results.

The verification was performed at NEC Corporation by applying the verification kit ‘Snapshot test kit(STK)’ provided in the White Paper, 'Guidelines for Using Snapshot Storage System for Oracle Database' published by Oracle Corporation and Oracle Storage Compatibility Program (OSCP).

* ‘NEC Storage’ is a series name of disk arrays from NEC Corporation for a world-wide market. For a Japanese market, ‘iStorage’ is applied for a series name such as ‘iStorage NV Series.’

2. Glossary

CIFS

A file-sharing protocol on Microsoft Windows

It is used primarily in Windows environments. However, its use on various UNIX OSs is increasing with the spread in the use of the Windows platform in recent years.

Cold Backup

Creating a backup copy of the production data while the production system is suspended state → Hot Backup.

File Lock

A mechanism that shows that a file is used by a specific process to avoid accidents such as double update.

(5)

state of the client. For example, if an NFS server reboots when a client locks a certain file, the state of the lock must be maintained after the reboot. The Oracle RDBMS uses the file lock mechanism to avoid, for example, starting the same instance twice. The NEC Storage NV Series is compliant with the RPC lockd mechanism commonly used on NFS.

Snapshot

Function to save the image of a file system at arbitrary time Consistent status of a file system can be saved per file system.

LVM

Logical volume Manager

Logical Disk Manager controls multiple logical disks made by the Disk Management function of the NEC Storage NV series as logical volumes.

Hard Mount (see also Soft Mount)

One of the mount options for NFS. A mode in which the client keeps I/O processing without breaking off, and waits for the file access system to recover, even if an error is caused during a file access.

Hot Backup

Creating a backup copy of the production data while the production system is operating

iSCSI (see also NAS, SAN)

One of the methods for connecting to a storage device over a commonly used network, such as LAN or Ethernet. FCIP and iFCP, and other methods which are similar to this, also exist. Unlike NAS, iSCSI is a technology that passes the device level SCSI protocol through the TCP/IP layer, and makes it possible to avoid the cost and management load for a special communication mechanism required by usual Fibre-Channel SAN.

Link Aggregation

Technology that improves bandwidth and fault tolerance by bundling two or more network lines (trunks) together and treating them just like one line. There is a standard based on IEEE803.2ad, and other individual standards used by network equipment manufacturers. The NEC Storage NV Series is compliant with IEEE802.3ad (LACP: Link Aggregation Control Protocol).

(6)

sophisticated applications, some devices are also capable of handling other protocols such as WebDAV, HTTP or FTP.

Note that sometimes even outside the field of data storage, in discussions of networks, ‘Network Access Server’ is known by the abbreviation NAS.

NFS

A common file protocol using TCP/IP defined by Sun Microsystems

As the specification is openly available, it is widely used on anything from the various UNIX platforms to the Windows system.

NTP

Network Time Protocol

The time synchronization protocol used on LANs or the Internet

NVRAM

Non-Volatile Random Access Memory

A generic name of memory that does not require an external power supply to retain information. Usually it is designed by using Static RAM or Flash ROM, etc. However, there are some defects with this approach such as difficulties in design, a high cost for large capacity (in the case of S-RAM) or low access speed (F-ROM).

SAN (see also iSCSI, NAS)

Storage Area Network

A technology that enables sharing from two or more host computers by treating storage as independent from a host computer when the storage capacity increases. Fibre-Channel is used for the connection. The host computer needs a connection and management mechanism to use Fibre-Channel.

Soft Mount (see also Hard Mount)

One of the mount options for NFS. A mode in which the client stops I/O processing if an error is caused during a file access.

Stale Cache

A state where data registered in a cache is older than what it replaces on the hard disk in the storage device.

As all of NEC Storage Series have advanced cache management mechanisms, a stale cache state is usually impossible.

Synchronous Writes

(7)

write is complete.

Conventionally, a storage device may execute a write to a disk with a time delay (Asynchronous Write) after reporting to a client that the write is complete, and receiving data into the main memory of the storage device (cache memory), in order to improve the apparent response performance. In such a procedure, if any trouble occurs after the recognition of write on the client side and before an actual write of data on the disk as “safely on” the storage device, a data loss which the client cannot be aware of would be generated.

In contrast, Synchronous Write enables the host to detect writes or losses of data at the time they happen by waiting to announce the completion of the write operation to a host side until the data is actually “safely on” the disk. The result is that reliability is improved in exchange for performance.

TOE

TCP/IP Offload Engine

A technology that provides high-speed TCP/IP processing and reduces the load on the OS by shifting the TCP/IP process out of the network processes of each layer, which the OS is usually responsible for, to the network interface and processing it with a hardware engine.

Some TOEs can support a higher-level application protocol. This type of technology specialized for a specific protocol (for example, iSCSI) is sometimes called a Network Accelerator.

(8)

3. Overview of NAS

3.1 What is NAS?

NAS (Network Attached Storage) is one form of a storage appliance that specializes in file server functions. Its features are described below.

3.1.1 Ease of Management

Conventionally, a file server is provided by configuring an external or internal storage such as a disk array coupled with a general-purpose host computer or server. Along with increasing storage capacity, networks known as SAN (Storage Area Networks) have become very popular. However, most of these systems need a special data transfer network using fiber channel or an independent management system, which causes an increase in management load. As NAS utilizes the commonly used Ethernet and common file protocols based on TCP/IP, which operates on the Ethernet, the management load can be greatly decreased. What is more, many engineers are familiar with Ethernet because it is a popular technology.

3.1.2 Storage

Consolidation

The file server described above naturally consolidates storage. That is, rather than each client having its own independent storage, many clients use the storage space concentrated on one site. The NAS system, which is specifically designed for this, has the following advantages compared with a file server configured with general-purpose servers and storage:

- Disks can be added while the system is on line and in use - The ability to provide file services to many clients

3.1.3 Multi-OS

Utilization

When using SAN for storage consolidation, common file processing and data sharing is difficult if the client OS differs, as the file access process is executed on the client.

With the NAS system, the NAS device controls the file system and data is offered to the client in the form of the file system. What is more, a common file process is available with a high degree of compatibility even if the OSs on clients are different, as de-facto standards such as NFS and CIFS are used as the access protocol to files.

3.2 NEC Storage NV Series

The NEC Storage NV Series has the following features in addition to the usual features of NAS.

3.2.1 High Performance TOE (TCP/IP Offload Engine)

Conventionally, the OS deals with TCP/IP and higher-level processes using resources such as CPU and memory. However, due to improvements in the transmission rate brought about by the development of the Ethernet technology, the situation now is that the proportion of communication processing required of the CPU has increased and application processes have become unavailable because of the heavy communication processing load.

The TOE technology is used to decrease the CPU load of the server and to achieve high performance and wide scalability by shifting the heavy load of TCP/IP processing from the software to the hardware. A technology specially designed for a specific application protocol (for example, NIC for iSCSI) is

(9)

sometimes called a Network Accelerator.

The NEC Storage NV Series is available to use two or more network interface cards each with a TOE chip on board and two Gigabit Ethernet ports. When a network interface card is added, the load for the increased network interface is managed by the TOE chip on the network interface card, which helps the network interface to reduce the load on the OS operation of the NAS device itself and brings about a high degree of scalability.

3.2.2 An Operating System highly optimized for NAS

To achieve a high-functionality, high-performance, high-scalability and high-availability NAS solution, the NEC Storage NV Series has highly optimized underlying software based on Linux.

3.2.3 High performance RAID Control LSI

The NEC Storage NV Series has achieved a high speed data transfer by applying high performance RAID processing LSI already proven in use in NEC Storage S-series disk arrays and Fibre-Channel HDD communicating at 2G bps on board.

3.2.4 Large capacity Non-Volatile RAM (NVRAM)

The NEC Storage NV Series makes use of large capacity non-volatile random access memory for its main memory, which greatly improves the performance compared with the standard speed of I/O response for writes to a disk. In addition, by configuring the batteries to use redundancy, a data loss in the memory can be prevented even if trouble occurs in one place.

3.2.5 Clusters

The NEC Storage NV Series has a model which can be used with a two-node cluster configuration that operates in Active-Active mode. (For these tests a single configuration model was used.)

3.2.6 Redundancy

The NEC Storage NV Series has redundancy built into its main parts such as the power supply, battery, fan and internal disk access paths.

In addition, clustering models in the NEC Storage NV Series also have server-level redundancy.

3.2.7 Data

Recovery

With the NEC Storage NV Series it is possible to make a snapshot for every file system available. A file that a user overwrote or deleted by mistake can be recovered by a simple operation from the client.

3.2.8 Optional

Functions

The NEC Storage NV Series can provide the following functions in addition to the above-mentioned basic functions:

- Remote replication - Backup

(10)

4. Snapshots on NEC Storage NV Series

An overview of the Snapshot functions provided on the NEC Storage NV Series NAS Device is described.

4.1 What is Snapshot Function?

The Snapshot function saves the image of a file system at arbitrary time. When an administrator creates a snapshot generation, the image of the file system at creation of the generation is saved. This feature enables reference to data before update processing, backup operation in online state, and data recovery by general users themselves.

4.2 Mechanism of the Snapshot Function

After creating a generation, the snapshot function works for the following when the file is updated: A new area is allocated for the data to be updated, and the updated data is written into the area. The data area before update processing in not updated, and the data is retained as a snapshot for reference. The data area is called “Snapshot area”.

The newly allocated area is updated until the next generation is created.

As explained above, old data and new data are saved in different areas. The old data area is shared until being updated, and the area occupied by the image of a generation is saved as a snapshot area and not released until the generation is deleted. Therefore, the file system area is incremented by the amount of updated data from the point of creating the generation.

4.3 Snapshot

Area

Management

Immediately after a generation is created, there is no area occupied by only snapshot generations. An area, which is occupied by only snapshot generation, is increased by only the size of an updated area each time data is updated from the point of creating a generation.

If snapshot generations are created while frequently updated data is stored in a file system, the difference between the generation data and the latest data becomes large and the number of areas occupied by snapshot generations increases.

The following functions are provided as snapshot area management functions for overall file system via NEC Storage Manager IP:

• Function to set a limit for areas occupied by only snapshot generations • Function to refer to the limit and utilization

When the number of areas occupied by only generations has reached the limit, a new generation cannot be created any more.

The number of areas occupied by only snapshot generations keeps increasing each time data is updated. Delete generations created by the snapshot function when they are no longer necessary.

(11)

4.4 Snapshot Operation

Snapshot commands used via ssh/rsh are as follows in briefly. Regarding setting information and detailed usage for each command, please refer to the related manuals.

• snap

Operate snapshot generations of logical volumes ¾ snap create

Create snapshot generation ¾ snap copyback

Recovery data by copying from snapshot generation ¾ snap delete

Delete snapshot generation ¾ snap list

Display snapshot generation listing

5. System configuration for OSCP

Hardware and software configurations applied for the verification tests on snapshot processing are shown here.

5.1 Hardware Configuration

Primary Oracle host/Secondary Oracle host

• Host : Sun Fire 280R(1CPU,1024MB memory,36.4GB x 2 local disk) • Storage : NEC Storage NV8210

NAS management host

• Host : NEC Express5800/100(IA server)

5.2 Software Configuration

Primary Oracle host/Secondary Oracle host • OS : Solaris 9 04/02

• Database : Oracle9iR2(9.2.0.5.0) • Storage : SC-LX OS Version 2.1.202 NAS management host

(12)

6. Summary of OSCP Test

Summaries of several verification tests results to utilize the backup data and to startup the replicated database on the OSCP test are shown here.

6.1 Backup and Restore

At first, create the snapshot generation of data file/table space, then copy back the backup data and finally check the Oracle database operates normally. Remote snapshot function is not subject to the test this time.

Location Backup Method Result

local cold √

hot √

remote cold out of validation hot out of validation

Table 6-1 : Backup and Restore

6.2 Startup of a replicated database

At first, create the backup of necessary files on snapshot generation, then copy the backup data from snapshot generation to logical volumes under the other server on which the replicated database runs. Startup Oracle database from the server and finally check the Oracle database operates normally.

Backup Method Result cold backup √ hot backup √

(13)

7. Details of the verification tests based on the OSCP

Test

In this chapter, details of the verification tests, its system configuration, the operation and some notes are described for each case of the table shown in the previous chapter with picking up an effective and typical combination of variations in the practical use. Also some extensions for each case are shown.

7.1 Database Construction for Verification and Snapshot Setting

7.1.1 Database Construction for Verification

Installation of Oracle Database

We installed the Oracle database according to the Oracle installation guide. In general, the values of parameters are as recommended, except that we modified some parameters.

Note that the same SID is used for verification of replicated database creation by dividing the execution server of the replicated database. (That is, we have not verified for the replicated database with another SID of the same host.)

7.1.2 Snapshot setting

In order to use the snapshot function of NEC Storage NV Series NAS Device, a user name on the NAS Device for rsh command execution from an operational server should be specified in advance. There are no settings for the operational server to execute rsh commands to the NAS Device.

7.2 Backup and Restore

The procedures for using the snapshot function on the NEC Storage NV Series NAS Device to back up the Oracle database are described. Additional enhancement method such that a snapshot generation is saved on a tape should be taken in actual operation. Of course, there is no problem to use the snapshot generation as a backup media.

In this validation test case, only one snapshot generation is used because the purpose is to check Oracle Database runs normally using the snapshot function.

(14)

System Configuration sample

Access from cx206 server to the NAS Device via NFS mount

Snapshot function

At the snapshot creation, the difference data between the generation data and the latest data are saved in the same volume. The following is LVM01 case.

NV8210-2

Access via NFS cx206

Primary Oracle host (SunFire 280R)

LVM01 LVM02

LVM03 LVM04

cx206 can use these volumes as normal operation or backup/recovery operation

datafile rbs datafile datafile

Create a generation as file images in the same volume by using snapshot function datafile rbs

LVM01

datafile datafile Normal operation files

(15)

Detailed volume usage

LD name usage mount point

LVM01 datafile/rbs/temp/system /export/oradata10 LVM02 control 1/Online REDO1 /export/oradata11 LVM03 control 2/Online REDO2 /export/oradata12 LVM04 init<SID>.ora/archived log /export/oradata13

7.2.1 Cold backup

This chapter explains the procedures for applying the snapshot function to perform cold backup processing. The following descriptions are for the cases assuming to apply snapshot function to backup data file located in a NFS mount volume. Other types of operations are also described as an extension.

1) Command operation

The following shows operation commands (create, delete, copy) to logical volumes in which data files exist for cold backup and their results.

1-1) Creating generation

Create a generation against a logical volume subject to snapshot

cx206:oracle > rsh -l cli nv8210-2 snap create /export/vol_euc/LVM01 snap snap 2004/08/12 17:39:54 created ok.

1-2) Deleting generation

Delete an unnecessary generation

cx206:oracle > rsh -l cli nv8210-2 snap delete /export/vol_euc/LVM01 snap sync snap 2004/08/12 16:44:54 removed

ok.

1-3) Recovering data from generation

Recover data from a generation which is saved in the same logical volume

(16)

2-1) Stopping the Oracle database

Cold backup is a backup solution while the database is not operating. Thus, stop the Oracle database. Issue the command from SQL*Plus.

cx206:oracle > sqlplus ‘/as sysdba’ SQL> shutdown immediate;

SQL> startup restrict pfile=/export/oradata13/initSNAP1.ora ; SQL> shutdown normal;

SQL> exit

Command output image is omitted here. (Similarly, it is omitted in the subsequent explanation if a note is not shown.)

The Oracle database has stopped when the SQL*Plus prompt is displayed after issuing the command. 2-2) Creating generation

For cold backup data, create a generation of a logical volume in which a data file exists. At this time, backup data can be saved to a tape device.

a) Delete an unnecessary generation if any

ok.

b) Create a generation. The generation can be saved to a tape device. cx206:oracle > rsh -l cli nv8210-2 snap create /export/vol_euc/LVM01 snap snap 2004/08/12 17:39:54 created ok.

2-3) Restarting the Oracle database

Place the database in normal operation status. Issue the command from SQL*Plus. cx206:oracle > sqlplus ‘/as sysdba’

SQL> startup open pfile= /export/oradata13/initSNAP1.ora SQL> exit

3) Use of cold backup data

Because this is a recovery of database, the original database is in halted state. 3-1) Recovering data from a generation

a) Delete the unnecessary generation named ”before_copyback” if copy-back was executed once cx206:oracle > rsh -l cli nv8210-2 snap delete /export/vol_euc/LVM01 before_copyback sync before_copyback 2004/08/11 17:03:45 removed

(17)

b) Copy a generation back and recover data

cx206:oracle > rsh -l cli nv8210-2 snap copyback /export/vol_euc/LVM01 snap creating "before_copyback" for safety

before_copyback 2004/08/12 17:43:51 created ok.

3-2) Recovering and restarting the Oracle database

By the operation in step 3-1, the contents of the volume containing the data file for the Oracle database are recovered to the backed-up point. To start the Oracle database, just apply the Online redo log and archived log in another volume to this data file.

Outline of the procedure for SQL operation

a) Start the database without opening.

b) Recover the database. (Apply the logs.)

c) Open the database.

Example in the verification test cx206:oracle > sqlplus ‘/as sysdba’

SQL> startup open pfile= /export/oradata13/initSNAP1.ora ; --- a) SQL> recover automatic database ; --- b) SQL> alter database open ; --- c) SQL> exit

In this verification test, because both Online redo log and Archived log are set to be securely stored in another volume, the database could be restored with minimum operation.

Extension 1) Cold backup and restore for the whole database

In addition to the above example, the following shows notes on using the snapshot function to back up all the related files in the database as cold backup:

Files to be backed up are:

Data file

Online Redo logs

Control files

(18)

The procedure for creating cold backup data is the same as the one for only the data file. Since the Oracle database is stopped, consistency among data file, Control files, Online Redo logs, Archived logs is maintained. Note that database restore for not only data files but the control files, Online Redo logs and archived logs means that the data is rolled back to the point when the backup data was created.

Recovery processing is not needed for restarting the restored database. (It is the same as to restart from the point when it stopped for backup data creation.)

7.2.2 Hot backup

1) Command operation

The following shows operation commands (create, delete, copy) to logical volumes in which data files exist for hot backup and their results.

Create a generation against a logical volume subject to snapshot

cx206:oracle > rsh -l cli nv8210-2 snap create /export/vol_euc/LVM01 snap snap 2004/08/12 17:39:54 created ok.

1-2) Deleting generation

Delete an unnecessary generation

ok.

1-3) Recovering data from generation

Recover data from a generation which is saved in the same logical volume

cx206:oracle > rsh -l cli nv8210-2 snap copyback /export/vol_euc/LVM01 snap before_copyback 2004/08/12 17:43:51 created ok.

2) Creation of hot backup data

2-1) Beginning the hot backup mode

Place each table space of the operating database in hot backup mode. Issue the command from SQL*Plus.

(19)

cx206:oracle > sqlplus ‘/as sysdba’ SQL> alter tablespace rbs begin backup; SQL> alter tablespace temp begin backup; SQL> alter tablespace system begin backup; SQL> alter tablespace snapdata begin backup; SQL> exit

For hot backup data, create a generation of a logical volume in which a data file exists. At this time, backup data can be saved to a tape device.

a) Delete an unnecessary generation if any

ok.

b) Create a generation. The generation can be saved to a tape device. cx206:oracle > rsh -l cli nv8210-2 snap create /export/vol_euc/LVM01 snap snap 2004/08/12 18:12:38 created ok.

2-3) Ending the hot backup mode

Displace each table space of the operating database from hot backup mode. Issue the command from SQL*Plus. Issue the command from SQL*Plus. If the snapshot function is applied, the hot backup mode need not to be continued until completion of backup processing for tapes, which makes it possible to minimize the time for hot backup mode. That is, influence on production would be minimized.

Issue the command from SQL*Plus. cx206:oracle > sqlplus ‘/as sysdba’

SQL> alter tablespace snapdata end backup; SQL> alter tablespace system end backup; SQL> alter tablespace rbs end backup; SQL> alter tablespace temp end backup; SQL> exit

Be sure to perform ‘end backup’ processing for all the table spaces that were placed in hot backup.

(20)

cx206:oracle > rsh -l cli nv8210-2 snap delete /export/vol_euc/LVM01 before_copyback sync before_copyback 2004/08/12 17:59:30 removed

ok.

b) Copy a generation back and recover data

cx206:oracle > rsh -l cli nv8210-2 snap copyback /export/vol_euc/LVM01 snap before_copyback 2004/08/12 18:19:01 created

ok.

3-2) Recovering and restarting the Oracle database

By the operation in step 3-1, the contents of the volume containing the data file for the Oracle database are recovered to the backed-up point. To start the Oracle database, just apply the Online redo log and archived log in another volume to this data file.

b) Recover the database. (Apply the logs.)

c) Open the database. cx206:oracle > sqlplus ‘/as sysdba’

SQL> startup open pfile= /export/oradata13/initSNAP1.ora ; --- a) SQL> recover automatic database ; --- b) SQL> alter database open ; --- c) SQL> exit

In this verification test, because both Online redo log and Archived log are set to be securely stored in another volume, the database could be restored with minimum operation.

7.2.3 Startup of a replicated database

Startup of a replicated database by using the snapshot data is available as one of the applications for a replicated database.

A replicated database is used for the followings:

Divide of production (such as load balancing by assigning the replicated database as a search-only database)

Simulation on the replicated database

Startup of a backup database

Procedure to startup a replicated database is as follows: 1) Creating backup data

(21)

2) Backup data of the database on a backup media.

3) Restart database operation.

2) Startup of the replicated database ( Followings are the operations on the production server for the replicated database )

1) Prepare the volumes on the replicated database.

2) Startup the replicated database.

The conventional method took much more time because the backup data was saved on tapes, etc. in step 2 in creating backup data. Also the method was not applicable and practical because of a huge amount of data. NEC Storage NV Series NAS Devices now provide the snapshot function for solving the conventional problems. The snapshot function enables creation of a replicated database with only a brief stop of production database or without stopping the production database by applying the hot backup function. And also NEC Storage NV Series NAS Devices provide a solution to startup the replicated database as a ‘copied’ database parallel to the production database. This chapter explains the procedures, and shows the results of the verification tests. System Configuration sample

Cx206 and 207 server access to the NAS Device via NFS mount.

cx207

Secondary Oracle host (SunFire 280R)

NV8210-2 cx206

Primary Oracle host (SunFire 280R)

Access via NFS mount

cx206 can use these volumes as normal operation or

LVM05

After Oracle data are copied form a generation to these volumes by cx206,

LVM08 LVM07 LVM06 LVM01 LVM04 LVM03 LVM02

(22)

generations to LVM05, LVM06, LVM07 and LVM08 and unmounts them. Then, cx207 server mounts LVM05, LVM06, LVM07 and LVM08 and startup the replicated database.

LD name usage mount point

LVM01 datafile/rbs/temp/system /export/oradata10

LVM02 control 1/Online REDO1 /export/oradata11

LVM03 control 2/Online REDO2 /export/oradata12

LVM04 init<SID>.ora/archived log /export/oradata13 LVM05 Data copied from a generation in LVM01 /export/oradata14 LVM06 Data copied from a generation in LVM02 /export/oradata15 LVM07 Data copied from a generation in LVM03 /export/oradata16 LVM08 Data copied from a generation in LVM04 /export/oradata17

Copy operation image

The following sample image shows Oracle data copy from LVM01 to LVM05. Generations created in LVM01 are copied (by copy command) to LVM05 via NFS.

Use this volume as replicated database

LVM0５

Copy created generations via NFS Creating generations when backup

LVM0１

7.2.3.1 Starting up a replicated database using cold backup processing

Stop the Oracle database, and create a replicated database by applying the snapshot function of the NEC Storage NV Series NAS device. Use of the snapshot function enables to minimize the time for stopping the production database.

It is easy and certain for the installation and settings of Oracle on the replicated database to be the same as on the production database. The procedure up to create the replicated database is the same as for the procedure for cold backup processing. However, note that not only data files but all the volumes of archived logs, control files, and Online Redo logs are also to be snapshotted.

(23)

The Oracle database SW (e.g., execution files) should be installed on the backup server beforehand. The snapshot function does not copy any files of the Oracle database SW.

Starting up a replicated database using cold backup processing

The same procedure as for creating cold backup data (already explained) is applied for creating backup data. Data files, archived logs, control files and Online Redo logs are to be snapshotted. The backup data is not applied for the copyback procedure, but is applied, after being copied to other logical volumes, as the operational volume on the other production server (backup server), which makes the Oracle database on the other production server to start up.

1) Creating backup data

1) Same as for normal cold backup procedure

2) Copying generations created in the original production server to logical volumes used in the other production server

1) Mounting logical volumes in the other operational server to the original operation server to copy generations created in the original operation server to logical volumes used in the other production server (should be done by super-user)

# mount /export/oradata14 # mount /export/oradata15 # mount /export/oradata16 # mount /export/oradata17

2) Copying generations created by the snapshot function at the cold backup time Copy generations created in the original server to logical volumes used in the other operation server.

cx206:oracle > cp /export/oradata10/.Snap/snap/*.* /export/oradata14 cx206:oracle > cp /export/oradata11/.Snap/snap/log*.ora /export/oradata15 cx206:oracle > cp /export/oradata12/.Snap/snap/log*.ora /export/oradata16 cx206:oracle > cp /export/oradata13/.Snap/snap/arc*.* /export/oradata17 cx206:oracle > cp /export/oradata13/.Snap/snap/*.ora /export/oradata17

3) Unmounting logical volumes used in the other operational server (should be done by super-user)

(24)

3) Starting up the replicated database

1) Mounting logical volumes used in the other operational server

# mount /export/oradata14 # mount /export/oradata15 # mount /export/oradata16 # mount /export/oradata17

2) Changing the path names of control files and archived logs included in init<SID>.ora to startup the replicated database

(berore)

control_files = ("/export/oradata11/ctl1snap1.ora", "/export/oradata12/ctl2snap1.ora") LOG_ARCHIVE_DEST = /export/oradata13

(after)

control_files = ("/export/oradata15/ctl1snap1.ora", "/export/oradata16/ctl2snap1.ora") LOG_ARCHIVE_DEST = /export/oradata17

Operation for database startup

If the paths for files on production database and the paths for those on replicated database are different because of the access from different servers, it is needed to change the path names of data file and Online Redo logs through SQL*Plus.

b) Change the path names of data file and Online Redo logs

c) Open the database.

Example in the verification test

SQL> startup restrict mount pfile=/export/oradata17/initSNAP1.ora --- a) SQL> alter database rename file '/export/oradata10/sys1SNAP1.ora' to

'/export/oradata14/sys1SNAP1.ora';

AQL> alter database rename file '/export/oradata10/rbs01.dbf' to '/export/oradata14/rbs01.dbf'; SQL> alter database rename file '/export/oradata10/temp.dbf' to '/export/oradata14/temp.dbf'; SQL> alter database rename file '/export/oradata10/snapdata.dbf' to

'/export/oradata14/snapdata.dbf';

SQL> alter database rename file '/export/oradata11/log1_1SNAP1.ora' to '/export/oradata15/log1_1SNAP1.ora';

(25)

--- b) SQL> alter database open; --- c) This sample shows that the replicated database can be started up independently.

7.2.3.2 Starting up a replicated database using hot backup processing

Create a replicate database by using the snapshot function of the NEC Storage NV Series NAS Device with the Oracle database in hot backup mode. Combination of the snapshot function and the hot backup function of the Oracle database enables to create the replicated database without stopping the production database.

It is easy and certain for the installation and settings of Oracle on the replicated database to be the same as on the production database. The procedure up to create the replicated database is the same as for the procedure for hot backup processing. However, note that not only data files but all the volumes of archived logs and Online Redo logs are also to be snapshotted.

The Oracle database SW (e.g., execution files) should be installed on the backup server beforehand. The snapshot function does not copy any files of the Oracle database SW.

Starting up a replicated database using hot backup processing

The same procedure as for creating hot backup data (already explained) is applied for creating backup data. Online Redo logs and archived logs are to be snapshotted as well as data files. The backup data is not applied for the copyback procedure, but is applied, after being copied to other logical volumes, as the operational volume on the other production server (backup server), which makes the Oracle database on the other production server to start up.

For the case of creating backup data including log files on hot backup mode, same as the case of creating backup data for overall database on cold backup mode, the database should be restored at a specific point. The specific point should be selected from the following 3 cases.

Apply whole archived logs that are effective until the last

Time specification

(26)

1) Creating backup data

1) Setting of hot backup mode

2) Snapshot for data files and table spaces 3) Terminate the hot backup mode

4) Specify the restore point and confirm the results of hot backup

Operation for backup data creation

The following explains the procedure for the case of applying whole archived logs that are effective until the last point. To confirm the archived log numbers, the procedure to switch logfile should be applied.

Consequently, it is sure to obtain all available archived log numbers.

a) Switch logfile.

b) Confirm the archived log number.

SQL> alter system switch logfile; ---a) System altered.

SQL> archive log list

Database log mode Archive Mode Automatic archival Enabled

Archive destination /export/oradata13

Oldest online log sequence 2501 ---b) Next log sequence to archive 2502

Current log sequence 2503 SQL> exit

2501 is the last archived log number to be obtained.

2) Copying generations created by the snapshot function at the hot backup time

1) Mounting logical volumes in the other operational server to the original operation server to copy generations created in the original operation server to logical volumes used in the other production server

2) Copying generations created by the snapshot function at the hot backup time 3) Unmounting logical volumes used in the other operational server

Copy procedures are the same as cold backup processing case but control files are not copied because control files will be created at the time of starting up the replicated database.

(27)

2) Changing the path names of control files included in init<SID>.ora The above two procedures are the same as cold backup processing case. 3) Starting up the replicated database by applying archived logs

If the paths for files on production database and the paths for those on replicated database are different because of the access from different servers, it is needed to change the path names of data file and Online Redo logs through SQL*Plus.

Control files should be recreated to startup the replicated database. And also RESETLOGS option should be specified at opening database.

a) Start the database on unmount status.

b) Create the control files for the replicated database.

c) Recover the database. (Apply the logs.)

d) Open the database specifying the RESETLOGS option.

cx207:oracle > sqlplus '/as sysdba'

SQL> startup nomount pfile=/export/oradata17/initSNAP1.ora ---a) SQL> create controlfile resetlogs database "SNAP1" archivelog ---b) 2 logfile group 1 ('/export/oradata15/log1_1SNAP1.ora','/export/oradata16/log2_1SNAP1.ora') size 1M, 3 group 2 ('/export/oradata15/log1_2SNAP1.ora','/export/oradata16/log2_2SNAP1.ora') size 1M, 4 group 3 ('/export/oradata15/log1_3SNAP1.ora','/export/oradata16/log2_3SNAP1.ora') size 1M 5 datafile '/export/oradata14/sys1SNAP1.ora', 6 '/export/oradata14/rbs01.dbf', 7 '/export/oradata14/temp.dbf', 8 '/export/oradata14/snapdata.dbf'; Control file created.

SQL> recover database until cancel using backup controlfile

ORA-00279: change 1809010 generated at 08/12/2004 15:56:23 needed for thread 1 ORA-00289: suggestion : /export/oradata17/arc0000002478.0001

(28)

ORA-00280: change 1809288 for thread 1 is in sequence #2479

ORA-00278: log file '/export/oradata17/arc0000002478.0001' no longer needed for this recovery

Data between the above and below is omitted.

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} (<RET>)

Specify log: {<RET>=suggested | filename | AUTO | CANCEL} cancel

Media recovery cancelled.

SQL> alter database open resetlogs; ---d) Database altered.

SQL> exit

As shown in the above, after applying the archived logs until 2501 (arc0000002501.0001) and operating CANCEL, then opening the database with the RESETLOGS option, the Oracle database is activated normally.

8. Reference

[1] N. Osorio, et. al., “Guidelines for Using Snapshot Storage Systems for Oracle Database,” Oracle Corporation, Aug. 2000.

Acknowledgement

We would like to thank Mr. Paul Tsien, Mr. J. Bill Lee and OCE support in Oracle Corporation for their helpful reviews and support for our OSCP project. We also would like to thank colleagues in our company associated with this project for their continuous support and advice.

NEC Storage NV Series