Scripting Storage Management Solutions with Hitachi Device Manager Software

(1)

Scripting Storage Management

Solutions with Hitachi Device

Manager Software

October 31, 2007

Craig Chan

(2)

Welcome to the WebTech Series

• Series of webcasts targeted specifically toward the more technical

user of Hitachi solutions

• Goals:

– Maximize the potential from your existing IT infrastructure

– Deploy best-in-class techniques for storage and storage management

– Benefit from real-life before and after scenarios

• Webcasts are once or twice a month – suggestions welcome for

upcoming sessions

(3)

What We Will be Covering

• How to download and install HiCommand® Command Line

Interface (CLI) for Device Manager

• Testing your installation with a few simple commands

• Editing the properties file to simplify commands

• Creating some batch scripts to execute on demand

• Using the system scheduler to run batch scripts periodically

(4)

Key Functions in the CLI

RefreshStorageArrays/AddStorageArray

These commands let you get the latest information from the arrays. To get the latest

information from all arrays, use RefreshStorageArrays. Be aware this will take a lot of time

for Device Manager instances with several arrays to manage. To get the latest information

from a specific array, use the AddStorageArray with a specific array to update the

information for that array only.

GetStorageArray

This command retrieves details about the storage subsystem already registered with the

HiCommand® Device Manager.

Array Group

Filter

FreeSpace

HostStorageDomain

Journal Pool

LDEV

LogicalDKC

LogicalUnit

Path

PDEV

Port

GetServerInfo

Returns information about your Device Manager installation including version information

and registered arrays.

(5)

More Functions in the CLI - Provisioning

AddHostStorageDomain

Creates a Host Storage Domain to add LUNs to.

AddWWNForHostStorageDomain

Adds world wide names from the host to ensure only that

host can access the LUNs.

AddLun

This command creates a path from the host to the volume

(logical unit). This command is use to allocate storage to

a host.

AddArrayReservation

Locks the array so other users can’t interfere with

resources you are about to use.

DeleteArrayReservation

Removes your lock on the array so other users can once

again manage the array. You should always do this after

you are done.

(6)

Getting Started – Installing the CLI

• Download the CLI package to any host

machine

(7)

Getting Started – Download Install Files

• Choose the installation

files for you platform

• The CLI requires the

JRE as well

– HDvM has specific

JRE requirements

which may not be

easily found on the

link provided.

– It’s often easier to

search for the right

JRE version online

instead.

• Install the files on the

host of your choosing

• Ensure communication

with the server is

available

– Check ports

(8)

Test a Command

• Try retrieving some information using GetServerInfo

C:\HiCommandCLI http://192.168.101.1:2001/service GetServerInfo –u system –p manager

RESPONSE:

An instance of ServerInfo

serverVersion=HiCommand Device Manager Server v5.8 Build 0580-03 (Aug 27, 2007) serverURL=http://localhost:2001

upTime=35 minutes 26 seconds

upSince=Thu, 25 Oct 2007 16:50:24 GMT currentApiVersion=5.8

license=0

List of 3 StorageArray elements: An instance of StorageArray arrayFamily=AMS/WMS displayArrayFamily=AMS/WMS An instance of StorageArray arrayFamily=USP displayArrayFamily=USP An instance of StorageArray arrayFamily=R600 displayArrayFamily=USP_V

(9)

Test Another Command

• Usually we refresh the storage array to make sure we have the latest information

C:\HiCommandCLI http://192.168.101.1:2001/service AddStorageArray ipaddress=172.16.100.1 family=USP –u system –p manager

RESPONSE: An instance of StorageArray objectID=ARRAY.USP.15050 [email protected] description=USP (15050) at 172.16.100.1 serialNumber=15050 arrayFamily=USP arrayType=USP microcodeVersion=50-09-00/00 agentVersion=04_09_00 productName=RAID500 controllerVersion=50-09-00-00/00 numberOfControllers=1 capacityInGB=7,847 cacheInMB=32,768 sharedMemoryInMB=-1 numberOfSpareDrives=-1 freeCapacityInGB=5,290 allocatedCapacityInGB=2,530

(10)

Too Much Typing

• Everything works, but notice we tend to type the same things over and over.

• We keep typing the URL to point to the server the commands are sent to.

• And we keep typing the username and password to perform the commands as.

C:\HiCommandCLI http://192.168.101.1:2001/service GetSystemInfo –u system –p manager

C:\HiCommandCLI http://192.168.101.1:2001/service AddStorageArray ipaddress=172.16.100.1 family=USP –u system –p manager

(11)

Editing the Properties File

• Edit

the HiCommandCLI.properties file in the CLI directory

• Make sure the URL points to your Device Manager Server

################################# #

# Device Manager Command Line Interface (CLI) Properties File #

# Can be used to provide options and default parameters for the # Device Manager CLI program.

#

# The Device Manager CLI program does not require any of the properties to be set.

<snip!>

# set the server url

#HiCommandCLI.serverurl=http://localhost:2001/service

##### OPTIONS ####

#user=khalsa

# password can be provided directly, or from a password file #password=khalsa

#password=@D:\\DeviceManager\\.passwd

<snip!>

(12)

Editing the Properties File

• Change the username to the one you want to execute the commands

• Change the password to correspond to that user.

################################# #

# Device Manager Command Line Interface (CLI) Properties File #

# Can be used to provide options and default parameters for the # Device Manager CLI program.

#

# The Device Manager CLI program does not require any of the properties to be set.

<snip!>

# set the server url

HiCommandCLI.serverurl=http://172.16.100.1:2001/service

##### OPTIONS ####

#user=khalsa

# password can be provided directly, or from a password file #password=khalsa

#password=@D:\\DeviceManager\\.passwd

<snip!>

user=system

password=manager

Note:

Passwords should not be included in clear text in this file. We are doing this for

(13)

Easier Commands

• Now we can type fewer parameters

– URL is always the same

– Username and password is always the same

C:\HiCommandCLI GetServerInfo RESPONSE:

An instance of ServerInfo

serverVersion=HiCommand Device Manager Server v5.8 Build 0580-03 (Aug 27, 2007) serverURL=http://localhost:2001

upTime=35 minutes 26 seconds

upSince=Thu, 25 Oct 2007 16:50:24 GMT currentApiVersion=5.8

license=0

List of 3 StorageArray elements:

C:\HiCommandCLI AddStorageArray ipaddress=172.16.100.1 family=USP RESPONSE:

An instance of StorageArray objectID=ARRAY.USP.15050 [email protected]

(14)

Creating Your First Script

• A common task is to refresh the Device Manager database so it is up to date with

the latest changes to the array.

REM This batch file refreshes all the arrays in Device Manager to make sure you REM have the latest information in your instance of Device Manager. This may take REM a long time to run so be aware of your environment

@ECHO OFF

ECHO Starting to refresh all arrays in HDvM. CALL HiCommandCLI RefreshStorageArrays

RefreshArrays.bat

• Then running the command is as simple as specifying the batch file name

C:\RefreshArrays.bat

Starting to refresh all arrays in HDvM RESPONSE: An instance of StorageArray objectID=ARRAY.USP.15050 [email protected] description=USP (15050) at 172.16.100.1 serialNumber=15050 arrayFamily=USP arrayType=USP

(15)

Extending Your First Script

• We should really perform a LUN Scan as well to update host and logical group

information with any new WWNs, but this script is more complicated

@ECHO OFF

ECHO Starting to refresh all arrays in HDvM.

CALL HiCommandCLI -o "allArrays.txt" RefreshStorageArrays

ECHO Updating hosts and logical groups for serialnum=15050 model=USP CALL HiCommandCLI AddLunScan serialnum=15050 model=USP

ECHO Updating hosts and logical groups for serialnum=77010000 model=AMS1000 CALL HiCommandCLI AddLunScan serialnum=77010000 model=AMS1000

ECHO Updating hosts and logical groups for serialnum=15060 model=R600 CALL HiCommandCLI AddLunScan serialnum=15060 model=R600

RefreshArrays.bat

• Now our script will update the host and logical group information as well

Starting to refresh all arrays in HDvM

Updating hosts and logical groups for serialnum=15050 model=USP

Updating hosts and logical groups for serialnum=77010000 model=AMS1000 Updating hosts and logical groups for serialnum=15060 model=R600

(16)

Something Complicated but Useful

• It would be nice to have a script that could just be dropped into your

environment and work instead of having to edit the file.

@ECHO OFF

CALL HiCommandCLI -o "allArrays.txt" RefreshStorageArrays FOR /F "tokens=1,2 delims==" %%i IN (allArrays.txt) DO (

IF "%%i"==" objectID" (

FOR /F "tokens=2,3 delims=." %%q IN ("%%j") DO (

ECHO Updating hosts and logical groups for serialnum=%%r model=%%q CALL HiCommandCLI AddLunScan serialnum=%%r model=%%q

)))

RefreshArrays.bat

• Now we can refresh Device Manager without updating this script

.

Starting to refresh all arrays in HDvM

Updating hosts and logical groups for serialnum=15050 model=USP

Updating hosts and logical groups for serialnum=77010000 model=AMS1000 Updating hosts and logical groups for serialnum=15060 model=R600

(17)

Even More Specifically

• Maybe you want to refresh only one array in Device Manager’s database

REM This batch file refreshes only the USP array in Device Manager, to make sure REM you have the latest information in your instance of Device Manager. Since we REM also know the serial number, we’ll refresh the hosts and logical groups as well. @ECHO OFF

HiCommandCLI AddStorageArray ipaddress=172.16.100.1 family=USP ECHO Updating the Hosts and Logical Groups

CALL HiCommandCLI AddLunScan serialnum=15050 model=USP

RefreshUSP.bat

• This command will take less time to execute since only one array is updated

C:\RefreshUSP.bat

Starting to refresh all arrays in HDvM RESPONSE: An instance of StorageArray objectID=ARRAY.USP.15050 [email protected] description=USP (15050) at 172.16.100.1 serialNumber=15050 arrayFamily=USP arrayType=USP microcodeVersion=50-09-00/00

(18)

Now For Something Useful

• It would be nice to ensure the Device Manager database was up to date each

morning

• Let’s take the RefreshArrays.bat script we created and schedule it to run every

night at 2:00 am

(19)

19

Provisioning Through CLI

• Let’s break down the tasks you need to perform

– Create a Host Storage Domain with the appropriate port

– Add world wide names to this Host Storage Domain

– Set the host mode settings to be specific to your environment

– Add logical units to your Host Storage

– Add these new storage paths to Logical Groups

• Here’s what it looks like in CLI commands

REM This batch file allocates storage to hosts. @ECHO OFF

ECHO Allocating storage to hosts.

CALL HiCommandCLI AddHostStorageDomain serialnum=15050 model=USP port=CL1-A domain=0x01 nickname=ExchServer1 hostmode=“Windows Extension”

CALL HiCommandCLI AddWWNForHostStorageDomain serialnum=15050 model=USP port=CL1-A domain=1 wwn=AA.AA.AA.AA.AA.AA.AA.AA

CALL HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:08

CALL HiCommandCLI AddObjectForLogicalGroup logicalpath=“/Windows/Server1/USP” groupelements=PATH.USP.15050.0.1.2568

CALL HiCommandCLI AddObjectForLogicalGroup logicalpath=“/Windows/Server1/USP” groupelements=PATH.USP.15050.0.1.2569

(20)

Performance of CLI

• CLI commands are executed serially

• This results in linear performance of the same AddLun command

• With CLI Batch, the commands are almost executed in parallel

• This results in a much faster response when allocating several LUNs

2500 2000 1500 1000 500 0

CLI

Time [sec]

CLI Batch

1 10 20 30 40 50 60 70 80 90 100

(21)

Provisioning Through CLI Batch

• First create a file to contain the batch commands (batch.txt)

beginbatch

HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:08 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:09 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:10 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:11 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:12 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:13 HiCommandCLI AddLUN serialnum=15050 model=USP port=CL1-A lun=0 domain=0x01 devnum=0A:14 HiCommandCLI AddLUN serialnum=15050 model=USP port=Cl1-A lun=0 domain=0x01 devnum=0A:15 endbatch

•

Then we’ll add LUNs to the host we previously defined.

REM This batch file allocates multiple LUNs to hosts.

REM It uses a faster process for multiple LUNs to provide quicker execution @ECHO OFF

CALL HiCommandCLI AddArrayReservation serialnum=15050 model=USP pause

REM If the array reservation fails, hit CTRL-C here to abort and try again

CALL HiCommandCLI –b batch.txt

(22)

What You Learned Today

• You should now be able to download and install HiCommand CLI for

Device Manager

• You can use the command line to manage your arrays

• You can create scripts to execute multiple commands

• You can schedule scripts to run periodically

(23)

References for the Material Today

• These scripts are available to you to use in your own

environment

– You will be able to find these scripts on the WebTech Portal

in both Windows Batch and Perl.

– Please contact John Harker at Hitachi Data Systems to have

these scripts sent to you.

• Additional details for the commands used today can be found in

the following guides:

– Command Line Interface (CLI) User’s Guide found on the

Device Manager CD

(24)

Upcoming WebTech Sessions:

We have several additional WebTech sessions planned this year

November 13:

Eliminating Redundant Data in Your Backup Environment

November 28

:

Best Practices for Oracle Automatic Storage Management and

Hitachi Replication Software

December 05:

Using Hitachi Dynamic Link Manager to Enable Server Boot from

the SAN

December 13: Effective Compliance Strategies – How to Protect Yourself

and Your Company

We will be adding additional sessions based upon participant requests.

Please go to www.hds.com/webtech

next week for the a link to the recording,

(25)

(26)

entire schedule and register at www.hds.com/webtech