roll-back the individual changes on a per-server or per chassis basis?

There is no uninstall/reverse functionality, although it is possible to force older components to overwrite newer ones if a reference set of the presently installed versions is created.

Note:

HP strongly discourages downgrading firmware unless absolutely necessary.

171

How often should we upgrade firmware?

Normally, update cycles should be set at approximately 6-month intervals. HP strongly recommends upgrading within the 9-month support window, which includes the currently released Quarterly Release Set plus two earlier versions to ensure vital updates and enhancements are installed. The result is optimal performance and availability of HP BladeSystem enclosures and servers.

How does the firmware version affect our support status with HP?

HP Support requires that when support is needed for a server, the serve must be aligned with one of the columns on the BladeSystem Firmware Compatibility Chart; otherwise, an administrator is

requested to update the server before additional support is provided. Doing so ensures that all known issues are addressed and possibly resolved and that the enclosure and blades are at a known support state before HP Support attempts to resolve the problem.

How do I add updated components to the HP BladeSystem FDT?

Today, the only want to update the FDT is to use UltraISO or another ISO editing tool that supports SLES10 SP2 (or in the SLES11) boot environments.

How do I add updated components to the HP firmware maintenance CD?

The following information will guide you in adding components to the HP Firmware Maintenance CD.

Using a USB Key

1. Insert a USB key containing the Firmware CD ISO image created.

If the blade server supports the SUV cable, use the HP USB Key Creator for Windows utility in the SUV (Serial-USB-Video) cable attached to the blade that is selected for update.

2. For Windows, download the HP USB Key Creator for Windows utility from the HP website and install it on a workstation.

Once installed, the utility places a shortcut in the Programs start menu folder of the HP System Tools.

Follow these steps to make your drive key bootable and copy the contents of the CD.

1. Place your HP USB drive key in an available USB port.

2. Select the HP USB Key Utility shortcut in the HP System Tools folder.

3. Complete each step presented by the application.

Note:

The HP USB Key Creator formats the USB key so that any files on the key are lost. Be sure to use a USB key that is at least 1 GB and does not contain any valuable files.

For Linux, you can manually create the USB key by following these steps.

1. Obtain SYSLINUX 3.2 or higher from http://freshmeat.com or http://syslinux.zytor.com from and download to a Linux workstation.

2. Install the SYSLINUX RPM obtained in Step 1.

3. Create a directory for the USB key mountpoint if one does not already exist. (For example, mkdir /usbkey.)

4. Insert the USB key and mount it (mount /dev/sda1 /usbkey).

172

Note:

The device mountpoint can vary depending on whether other SCSI drives are also installed on the server. Thus, the device mountpoint can be sdb1, sdc1, etc).

5. Issue the command ./syslinux /usbkey to have SYSLINUX write out the boot partition to the USB key.

Caution:

Failure to follow this step could lead to a key that will not boot the Firmware CD until the SYSLINUX command is successful.

6. Create a directory to mount the CD image.

mkdir /cd_mount_point

7. Insert the Firmware CD or mount the Firmware CD ISO via a loopback.

mount /dev/cdrom /cd_mount_point or

mount -t iso9660 firmware-<version>.iso /cd_mount_point -o loop 8. Change to the /usb directory on the CD.

cd /cd_mount_point/usb

9. Execute the usbcreator.sh shell script passing in the CD mount point and the USB mount point to move the Firmware CD files to the USB key.

./usbcreator.sh /cd_mount_point /usbkey

10. If additional components must be added to the USB, copy them into the /compaq/swpackages (Linux) or the \compaq\swpackages (Windows) directory and HP SUM picks them up

automatically if the HP SUM version being used can support the type of components added.

11. .Unmount the CD and the USB key.

umount /dev/cdrom umount /dev/sda1

Note:

This must match the initial mount point in Step 4.

12. Remove the USB key and CD from the workstation used to create the USB key.

Using a hard drive

1. Copy the contents of the \compaq\swpackages directory from the CD or ISO image to a directory on the hard drive in the location where HP SUM will be executed.

2. Ensure execute privileges are available in Linux by using the chmod 700 * command. By default, the files are copied off the CD in Linux as read-only with no execution privileges.

3. Copy any updated files into the same directory as where the files are copied in Step 1.

4. Execute HPSUM so that the new firmware components are recognized.

173

How do virtual environments affect the firmware update process?

Unfortunately, none of HP current firmware update tools work in virtualized environments. The problem is the Virtual Machines abstract the physical hardware and firmware updates deal with physical hardware. VM’s have limited ability to access underlying hardware through vendor-specific APIs. Because of the physical hardware abstraction, it is necessary for the user to either shut down or move all of the VMs to another server and reboot the server into another boot environment, such as the ones provided by the HP BladeSystem FDT and the HP Firmware Maintenance CD. These boot environments provide access to the physical hardware so the firmware can be updated. Once the update process has been completed, the virtual machine environment can be restored to its initial configuration by either restarting the VMs or moving them back to the recently updated server.

Installed firmware version data collection

Enclosure-level firmware information

Unfortunately, there is not a single tool to pull all firmware-version information for an enclosure.

However, by scripting the use of the available tools, it is possible to pull together a list of much of the firmware in an enclosure.

Through a combination of SSH scripts and RIBCL commands, it is possible to pull the OA version, the iLO firmware version, the System ROM version, PowerPIC, and the VC firmware version.

There are a number of possible ways to collect the firmware revision date of the current install base.

HP recommends using the HP SIM tool, especially when collecting information from a large IT environment. This is because both standard and customized report can be derived from the tool that enables viewing of the various firmware revisions that might be present.

Another way to collect firmware information is by using a CLI approach when connecting to the OA with OpenSSH. Once SSH has been installed, OA CLI commands can be issued against the active OA to obtain firmware information about the enclosure. The OA CLI commands are documented in the HP BladeSystem Onboard Administrator Command Line Interface User Guide.

Use the ssh command to retrieve the OA firmware as follows:

ssh -l Administrator 16.89.1.3 show oa info

In this command, Administrator is the OA administrator account and 16.89.1.3 is the address of the active OA module. The output from the command returns a similar output as the following example.

The highlighted element contains the version of OA firmware currently installed (2.52).

Onboard Administrator #1 information:

Product Name : BladeSystem c3000 Onboard Administrator Part Number : 448589-B21

Spare Part No.: 441832-001 Serial Number : P3059X39VUE073 UUID : 18P3059X39VUE073 Manufacturer : HP

Firmware Ver. : 2.52 Jul 13 2009 Hw Board Type : 1

Hw Version : X3

The ssh command can be used to retrieve the installed versions of iLO, System ROM, and PowerPIC firmware by using the OA show server info all command. The highlighted areas reflect the System ROM version, the iLO firmware version and the PowerPIC firmware version in sequence.

ssh -l Administrator 16.89.1.3 show server info all Server Blade #1 Information:

Type: Server Blade Manufacturer: HP

174

Product Name: ProLiant BL280c G6 Part Number: [Unknown]

System Board Spare Part Number: 531337-001 Serial Number: 2UX91700UU

UUID: 37373035-3738-5532-5839-313730305555 Server Name: WIN-P2L0THRBZYQ

In some cases, the Power Management controller firmware version shows Not Available. This is usually an indication that the Power Management controller is not functioning correctly. If this

happens, try powering off the blade, removing it from the enclosure, and re-inserting it. If the problem still occurs, contact HP Technical Support to report the problem.

Management Processor Information:

To find the version of installed firmware for the VC, use the show firmware command by using OpenSSH for Windows or the ssh RPM for Linux as shown in the example that follows. The password must be provided, unless SSH credentials are established with the VCM.

C:\>ssh -l Administrator 16.83.61.67 show firmware Password:

In this example, the VC Ethernet modules in Bay 1 and Bay 3 are 2.12 of VC firmware, and the VC Fiber Channel module in Bay 5 is 1.32.

175 Blade server-level firmware information

As with the Enclosure-Level firmware information, there is not a single tool that provides all firmware version information within a blade. However, by leveraging data reported by the individual smart components, most of the installed firmware versions can be obtained without too much effort. The only exception is the installed firmware version for the Emulex, Brocade, and QLogic Fiber Channel HBAs.

Currently, the firmware version for the HBAs can only be obtained in Linux without the use of external applications such as SANSurfer, CommandView, and others.

This section covers how to obtain the installed firmware versions for the following firmware types:

NIC firmware

To determine the versions of NIC firmware installed in a given server, the NIC firmware smart component must be available. The NIC firmware component provides its own firmware version information for each type of firmware available for each NIC found in the server. The component can be executed in discovery mode and provides a file name to generate the special discovery output file.

The NIC firmware component can be executed in discovery mode by using the following command.

CP010605.scexe /d CP010605_nic.xml Where:

 CP010605.scexe is the name of the NIC firmware component

 /d is the discovery option

 CP010605_nic.xml is the output file name to write the discovered firmware information

Note:

The /d option must be /d and must immediately be followed with a file name. Otherwise, the firmware component launches the NIC firmware installers instead of identifying the available NIC firmware.

The resulting output file shows the information in the following example. In this example, the output has been pruned to show only necessary details.

…

<alt_name value="HP NC-Series Broadcom Online Firmware Upgrade Utility for Linux" />

<takes_effect value="reboot" />

<device_id value="eth0" />

…

<product_id value="HP NC373i Multifunction Gigabit Server Adapter" />

… <fw_item>

…

<active_version value="1.9.6" />

… </fw_item>

<fw_item>

…

<active_version value="n/a" />

… </fw_item>

176

<fw_item>

…

<active_version value="1.2.2" />

… </fw_item>

</device>

</devices>

…

The first version field located near the top of the returned XML file (2.1.8 in the example) indicates the version of the component being used and does not indicate the individual firmware types.

Each NIC has its own <device> entry. The entry also includes product name and MAC address of the adapter that is being reported, plus multiple firmware items (fw_item) entries that indicate the

installed firmware version for each type of firmware available on the NIC. For each type of firmware, the <version> field indicates the version of firmware that is deployed if the firmware update is initiated with this component. The <active_version> field indicates the currently installed version of this firmware.

In the previous example, the Bootcode shows the installed version in the <active_version> field as 1.9.6 and the version available in the component as also being 1.9.6. In the PXE item, the n/a indicates this NIC has PXE support disabled and, therefore, cannot be updated. In the last firmware item for iSCSI support, the installed version of firmware is 1.2.2 and the component updates it to 1.2.3 if an update was attempted with the CP010605.scexe component used to generate the discovered data.

Note:

The same discovery file format is also available using the Windows smart components.

Tape firmware

To determine the version of tape firmware installed on a given tape drive, the tape firmware smart component must be available. The tape firmware component provides its own firmware version information for each tape drive found in a server. The component can be executed in discovery mode and provides a file name to generate the special discovery output file. The tape firmware component can be executed in discovery mode by using the following command.

CP010445.scexe -s -d CP010445_tape.xml Where:

 CP010445.scexe is the name of the tape firmware component

 -s is the silent option

 -d is the discovery option

 CP010445_tape.xml is the output filename to write the discovered firmware information

The CP010445_tape.xml file option must be passed immediately after the -d option for this to work.

This file can then be parsed to obtain the drive type and firmware as highlighted in the following sample tape firmware discovery output file called CP010445_tape.xml in our example.

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>

<hp_rom_discovery version="2.0.0.0">

<alt_name value="HP TAPE Firmware for Drive x, Drive y, and Drive z for Windows"/>

177

<takes_effect value="immediate"/>

In this example, the Ultrium 4 SCSI tape drive in the server is 11.825000 of the firmware that is available for that type of drive.

Note:

The same discovery file format is also available by using the Windows smart components.

Smart Array controller firmware and SAS/SATA hard drive firmware versions

The HP Array Configuration Utility Command-Line Interface program (hpacucli) can pull controller and hard-drive firmware information. The latest version of the hpacucli utility is found in the latest HP ProLiant Support Pack version. Once installed, the binary file and its support libraries are placed in the C:\Program Files\Compaq\Hpacucli\Bin” or “C:\Program Files

(x86)\Compaq\Hpacucli\bin directory, depending on the architecture of the system on which it was originally installed. The files in the directory can be scraped and added to a script that is executed on each server. Doing so pulls out the installed firmware version for the Smart Array Controllers and hard drives located behind these controllers in a given server. The following is an example of the command to show the details for

C:\Program Files\Compaq\Hpacucli\Bin>hpacucli ctrl all show config detail Smart Array P400 in Slot 1

…

178

Drive Type: Data Drive Interface Type: SAS Size: 36 GB

Rotational Speed: 10000 Firmware Revision: HPD7

Serial Number: 3LC04GYB00007543CG6U Model: HP DG036A8B53

…

In this example, which has been reduced to show only the relevant parts, the Smart Array firmware 5.26 and of the drives shown, one is HP model number DG036A8B5B and has firmware HPD4, and the other is HP model number DG036A8B53 and has firmware HPD7 installed. Multiple controllers and drives can be reported in a single call to hpacucli.

Note:

The same functionality also exists for Linux with the HP array configuration utility command-line interface RPM provided in the ProLiant Support Packs for Linux.

Troubleshooting

Troubleshooting HP SUM

When using HPSUM, the logs files are located at the following locations:

 Windows (for both firmware and drivers)

%SYSTEMDRIVE%\cpqsystem\hp\log\<ip_address>\hpsum_detail_log.txt

 Linux

/var/hp/log/<ip_address>/hpsum_detail_log.txt

Note:

Using the offline environments provided by the Firmware Maintenance CD and the HP BladeSystem FDT result in the log files being written to the Linux location.

HP SUM also creates trace logs of its installation that can be used to debug problems. These trace files are located in the following places:

 Windows Server 2003/XP/Vista:

%TEMP%\hp_sum\

 Windows Server 2008:

%TEMP%\<number>\hp_sum normally the <number> is “1”

 Linux:

/tmp/hp_sum

HP SUM creates a separate trace log for each target server. These are captured in subdirectories under the trace log directories noted in this example. The directories start with Target_0, Target_1, and so on. These directories contain the target specific trace information. No passwords or sensitive data is stored in this trace logs.

179

How to debug Discovery failures

If you receive an HPSUM Connection Error or Discovery Failed messages, there are several troubleshooting tips.

1. Ensure your workstation does not have an existing connection to the ADMIN$ share on the target IP address. If it does, it prevents HP SUM from connecting to the remote server share, because Windows only allows one connection from a client to a server share. This can be checked by entering net use‖ at a command prompt. If there is an existing share to the target IP address \admin$ share, delete it and try the installation again.

2. Ensure the target IP address server admin$ share is accessible. To accomplish this, validate the target server can be accessed by entering net use x:

\\<ip_address_or_dns_name>\admin$ for the target server IP address or DNS name. Once the connection is validated, ensure it gets deleted by entering net use x: /d at a command prompt.

3. Ensure that the user id for connecting to the target IP address server is part of the administrators group. If it isn’t, HP SUM blocks installation to the target.

4. Ensure WMI is enabled and running on all Windows target servers 5. For Linux, ensure the SSH port is not blocked.

In some rare cases, external storage enclosures may cause HP SUM to hang or report discovery failed. To resolve this problem, disconnect the external storage until the firmware updates are complete.

For Linux, ensure the target server can be contacted through SSH and that the scp command is available to securely send files to the target server.

Ensure the firewall ports on any routers in the network, as discussed in HP SUM Special Considerations.

The Symantec End Point Protection product (SEP) blocks HP SUM ability to communicate with remote targets if the Network Threat Analysis feature is enabled. The only workaround so far is to disable this

In document HP BladeSystem ProLiant Firmware Management Best Practices (Page 170-188)