Exercise instructions with hints
Part 1: rootvg disk replacement
Hypothetical scenario:
While you have your application data on SAN disks, your rootvg is on a disk in a disk bay which is integrated into your server. Recently, you have noticed frequent temporary but recoverable disk error events (DISK_ERR4) in the AIX error log. You have decided to replace the disk, but you do not want to take down the system. There is already a spare disk in the disk bay.
You will migrate the rootvg content from the failing disk to this spare disk and then replace the failing disk. (This has scared you enough that you plan to mirror the rootvg when the bad disk is replaced).
__ 1. Check to see if hdisk1 is assigned to a volume group, If it is, remove it from that volume group.
» Example commands for doing this: » # lspv
» If needed:
- -- for VG with another disk --- # migratepv hdisk1 hdisk# # reducevg <VGname> hdisk1 OR
- -- for VG with only hdisk1 --
#reducevg -d <VGname> hdisk1
__ 2. Extend the rootvg volume group to include the hdisk1 physical volume. You might have to use the force flag if there is an old VGDA on the disk.
EXempty » # extendvg rootvg hdisk1
__ 3. Migrate only the rootvg boot logical volume from hdisk0 to hdisk1. » # migratepv -l hd5 hdisk0 hdisk1
__ 4. Regenerate the contents of the boot logical volume. » # bosboot -ad /dev/hdisk1
__ 5. Clear the old boot record of hdisk0. » # chpv -c hdisk0
__ 6. Update the bootlist to only try to boot off of hdisk1. » # bootlist -m normal hdisk1
__ 7. Migrate all rootvg logical volumes, remaining on hdisk0, to hdisk1. The migration will take a few minutes; be patient.
» # migratepv hdisk0 hdisk1
__ 8. Verify that all of the logical volumes have been moved from hdisk0 to hdisk1. Then, remove the hdisk0 physical volume from the rootvg volume group.
» Example commands are: # lspv -l hdisk0 # lspv -l hdisk1
# reducevg rootvg hdisk0 # lsvg -p rootvg
__ 9. Delete hdisk0 from the ODM. » Example commands are:
# rmdev -d -l hdisk0 # lsdev -c disk
__ 10. We will assume that the failing disk has been replaced through a hot swap procedure. Rediscover and configure the replacement disk.
» Example commands are: # cfgmgr
# lsdev -c disk
__ 11. Later exercises assume that the rootvg resides on hdisk0. Migrate the rootvg back to hdisk0. The migration will take a few minutes; be patient
» Example commands for doing this: # extendvg rootvg hdisk0
# migratepv -l hd5 hdisk1 hdisk0 # bosboot -ad /dev/hdisk0
# chpv -c hdisk1
# bootlist -m normal hdisk0 # migratepv hdisk1 hdisk0 __ 12. Remove hdisk1 from the rootvg. » Example command for doing this: » # reducevg rootvg hdisk1
EXempty
Part 2: User VG disk replacement procedure
__ 13. The lvmtestvg volume group should not exist. If it does exist, remove it. » An example command is:
# reducevg -d lvmtestvg hdisk2
__ 14. Use the /home/AN14/build_vg script to create the volume group. » # /home/AN14/build_vg
__ 15. List the logical volumes in lvmtestvg. The /lvmtestfs filesystem should be open (mounted).
If /lvmtestfs is defined, but not mounted, then mount it at this point. » An example command is:
» # lsvg -l lvmtestvg
» If /lvmtestfs is not mounted (not open): » # mount /lvmtestfs
__ 16. Create some files in the /lvmtestfs file system by copying configuration files from the /etc directory.
» An example command is:
» # cp /etc/*.conf /lvmtestfs
We will pretend that you have the following situation:
You have your user data in the SAN. The disk arrays were purchased mainly on a criteria of lowest price. The disk arrays have had performance and functional problems, resulting in the purchase of a more reliable replacement disk array. You have been assigned to (non-disruptively) move the user volume group off of the old disk and onto the disk that is backed by a LUN in the new disk array.
__ 17. Check to see that another disk is available (such as hdisk3). » An example command is:
» # lspv
__ 18. Extend the volume group to include the extra disk. » An example command is:
» # extendvg lvmtestvg hdisk3
__ 19. Migrate the data which is on the failing disk to the new disk. » An example command is:
» # migratepv hdisk2 hdisk3
» An example command is: » # lspv -l hdisk2
__ 21. Verify that the data you created is still there. » An example commands is:
» # ls -R /lvmtestfs
__ 22. Remove the old disk from the VG and verify that your volume group now has only the new disk.
» Example commands are:
» # reducevg lvmtestvg hdisk2 » # lsvg -p lvmtestvg
__ 23. Remove the old disk from the ODM customized device database and verify that it has been deleted from the ODM.
» Example commands are: »# rmdev -dl hdisk2 » # lsdev -Cc disk
__ 24. At this point, we will assume that the SAN administrators have created and zoned a new LUN for our system. Discover and configure the disk. Verify that we now have an hdisk2 disk.
» Example commands are: » # cfgmgr
» # lsdev -Cc disk
__ 25. Finally, let us cleanup what we have created in this part of the exercises.
Remove the lvmtestvg volume group. You will either need to first remove any logical volumes or request that the removal of the logical volumes be handled as part of removing the volume group.
» Example commands are: » # umount /lvmtestfs
» # reducevg -d lvmtestvg hdisk3 Note
EXempty
Part 3: (Optional) Working with quorum
__ 26. Create a volume group named newvg using one unused disk. » Check to see which disks are free:
» # lspv
hdisk0 00c07f7f59ac0ea9 rootvg active hdisk1 00c07f7fbcdf4791 None
hdisk2 00c07fbf59f134be None hdisk3 none None
» Create the volume group with one disk: (We will assume that this disk is hdisk2 in the rest of the exercise examples.)
»# mkvg -y newvg hdisk2
__ 27. Use the lsvg command to find the volume group information for the newvg volume group.
a. Quorum: ________
b. Number of VGDAs (VG Descriptors): ____________ c. Active physical volumes: ___________
» Command and sample output: »# lsvg newvg
VOLUME GROUP: newvg VG IDENTIFIER: 00c07f7f00004c00000001218da898f3
VG STATE: active PP SIZE: 128 megabyte(s) VG PERMISSION: read/write TOTAL PPs: 546 (69888 megabytes) MAX LVs: 256 FREE PPs: 546 (69888 megabytes) LVs: 0 USED PPs: 0 (0 megabytes) OPEN LVs: 0 QUORUM: 2 (Enabled) TOTAL PVs: 1 VG DESCRIPTORS: 2
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 1 AUTO ON: yes MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32 LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
» Quorum: 2 » 2 VGDAs
» 1 active physical volume
__ 28. Add a second unused disk to the newvg volume group.
» We will assume that this disk is hdisk3 in the rest of this exercise. » # extendvg -f newvg hdisk3
__ 29. Use the lsvg command to find the volume group information for the newvg volume group.
a. Quorum: ____________________ b. Number of VGDAs: ____________ c. Active physical volumes: ___________ » Command and sample output:
» # lsvg newvg
VOLUME GROUP: newvg VG IDENTIFIER: 00c07f7f00004c00000001218da898f3
VG STATE: active PP SIZE: 128 megabyte(s) VG PERMISSION: read/write TOTAL PPs: 625 (80000 megabytes) MAX LVs: 256 FREE PPs: 625 (80000 megabytes) LVs: 0 USED PPs: 0 (0 megabytes) OPEN LVs: 0 QUORUM: 2 (Enabled) TOTAL PVs: 2 VG DESCRIPTORS: 3
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 2 AUTO ON: yes MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32 LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
» Quorum: 2 » 3 VGDAs
» 2 active physical volumes
» The number of PVs and the number of VGDAs both increased.
__ 30. Use the lspv command to identify how many VGDAs there are on each disk in the VG.
• Number of VGDAs on original PV for newvg (hdisk2): ____________ • Number of VGDAs on additional PV for newvg (hdisk3): ____________ » Commands and sample output:
»# lspv hdisk2 | grep -i descriptor
TOTAL PPs: 546 (69888 megabytes) VG DESCRIPTORS: 2
»# lspv hdisk3 | grep -i descriptor
TOTAL PPs: 79 (10112 megabytes) VG DESCRIPTORS: 1
» 2 VGDAs on hdisk2 » 1 VGDA on hdisk3
EXempty __ 31. Make the second disk (with only one VGDA) unavailable using the following steps. __ a. Varyoff the newvg volume group.
»# varyoffvg newvg
__ b. Make the second disk unavailable using rmdev. Do not delete it from CuDv, just change the device state from available to defined.
» Use rmdev -l <disk>. Do not use the rmdev -d flag. »# rmdev -l hdisk3
hdisk3 Defined
__ 32. Try to vary on the newvg volume group. Did it vary on? _____________________ What is the status of the disk you unconfigured? __________________________ » Command and sample output:
»# varyonvg newvg
PV Status: hdisk2 00c07fbf59f134be PVACTIVE hdisk3 00c07f7f8db00776 PVMISSING varyonvg: Volume group newvg is varied on.
» The volume group did vary on, but the second disk was missing. __ 33. Look in the error log file to see if any errors were logged.
» # errpt -A | pg
--- LABEL: LVM_QUORUMNOQUORUM
Date/Time: Fri May 29 15:09:19 2009 Type: INFO
Resource Name: LIBLVM Description
Activation of a no quorum volume group without 100% of the disks. Detail Data
MAJOR/MINOR DEVICE NUMBER 0021 0000
SENSE DATA
00C0 7F7F 0000 4C00 0000 0121 8DA8 98F3 0000 0000 0000 0000 0000 0000 0000 0000 --- LABEL: LVM_MISSPVADDED
Date/Time: Fri May 29 15:09:19 2009 Type: UNKN
Resource Name: LIBLVM Description
PHYSICAL VOLUME DEFINED AS MISSING Detail Data
MAJOR/MINOR DEVICE NUMBER 0011 0003
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000
---
__ 34. Bring the second disk in the newvg volume group back to an available state and verify the state by listing the device.
» Use cfgmgr to bring hdisk3 on-line. »# cfgmgr
OR
»# mkdev -l hdisk3
» Use lsdev to list the state. »# lsdev -l hdisk3
hdisk3 Available Virtual SCSI Disk Drive
__ 35. Display the physical volumes in the newvg volume group. What is the PV STATE of the second disk?
_______________________________________________________________ » Commands and sample output:
»# lsvg -p newvg
newvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION hdisk2 active 546 546 110..109..109..109..109 hdisk3 missing 79 79 16..16..15..16..16
EXempty » The second disk (hdisk3) is still in a missing state.
__ 36. What do you think will bring hdisk3 into an active state in the newvg volume group? Try your strategy. (Look at the Hints if you do not know.)
Verify that it worked by running the lsvg -p newvg command. » Running the varyonvg command works.
» Note: You do not have to vary it off first. you can run varyonvg on a volume group that's
already active. This will refresh the state of the disks in the volume group. » # varyonvg newvg
» # lsvg -p newvg
newvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION hdisk2 active 546 546 110..109..109..109..109 hdisk3 active 79 79 16..16..15..16..16
» The second disk, hdisk3, is now back in an active PV state.
In the previous steps, you removed the second disk (which only had one VGDA). In the following steps, you will remove the first disk (which has two VGDAs).
__ 37. Make the first disk unavailable using the following steps. Varyoff the newvg volume group.
»# varyoffvg newvg
__ 38. Make the first disk unavailable using rmdev. Do not delete it from CuDv.