LED 553 halt

This section sets out a procedure to recover from an LED 553 and applies to AIX Version 4 and AIX 5L Version 5.1.

An LED 553 occurs during IPL on a RS/6000 if the system cannot read or run the /etc/inittab file.

To recover from an LED 553, check /dev/hd3 and /dev/hd4 for space problems and erase files, if necessary. Check the /etc/inittab file for corruption and fix it, if necessary. If the inittab file was not corrupted, you will need to check the shell profiles, the /bin/bsh file, and some other files:

1. Turn the key to the Service position (MCA machines), then power on. On PCI machines, power on and press F5 or 5 when the Icon screen appears. This will start the Service mode boot from CD-ROM or mksysb.

2. With bootable media of the same version and level as the system, boot the system. The bootable media can be any one of the following:

– A bootable CD-ROM or DVD – mksysb

– A bootable install tape

Follow the prompts to the Welcome to Base OS menu.

3. Choose Start Maintenance Mode for System Recovery (option 3). The next screen contains prompts for the Maintenance menu.

Choose Access a Root Volume Group (option 1). At this stage, the console displays a warning that indicates you will not be able to return to the Base OS menu without rebooting. Choose 0 and press Enter.

At this stage, the console will display information about all the volume groups on the system. rootvg will usually be the top one in the list. Choose 1 and the console will display the logical volumes contained within the chosen volume group. If it does not appear to be rootvg, take the option to return and choose another volume group. When rootvg has been found, choose Access this volume group and start a shell before mounting file systems (option 2).

If you get errors from the preceding option, do not continue with the rest of this procedure. Correct the problem causing the error. If you need assistance to correct the problem causing the error, contact your AIX support personnel.

4. Use the df command to check for free space in /dev/hd3 and /dev/hd4:

# df /dev/hd3

# df /dev/hd4

If df shows that either file system is out of space, erase some files from that file system. Three files you may want to erase are /smit.log, /smit.script, and /.sh_history.

5. Check the /etc/inittab file for corruption. It may be empty or missing, or it may have an incorrect entry. For comparison, see the sample inittab file shown in Example 3-1 on page 66. Set your terminal type in preparation for editing the file by setting the TERM environment variable. For example:

# TERM=xxx

# export TERM

Where xxx stands for a terminal type, such as lft, ibm3151, or vt100.

Now, use an editor to create the /etc/inittab file. It may be possible for you to correct a corrupt inittab. If the file is missing, or it is not possible to repair the corruption, it is simpler to create a new file. If your /etc/inittab file was corrupt, and you recreated it, you may not need to perform any of the following steps.

6. Use the following command to check for any modifications or problems with permission:

# ls -al /.profile /etc/environment /etc/profile Here is an example output:

-rw-r--r-- 1 root system 158 Dec 14 2001 /.profile -rw-rw-r-- 1 root system 1389 Oct 26 2001 /etc/environment -rw-r--r-- 1 root system 1214 Jan 22 2001 /etc/profile

7. /etc/profile or .profile may contain a command that is valid only in the Korn shell. Change the command to something that is also valid in the Bourne shell. For example, change the following:

export PATH=/bin:/usr/bin/:/etc:/usr/ucb:.

to the following:

PATH=/bin:/usr/bin/:/etc:/usr/ucb:.

export PATH

8. /etc/environment is a special case. The only commands it may contain are simple variable assignments, such as statements of the form var=value.

Check this file with an editor to verify the format.

9. Check for missing or moved files with the following command:

# ls -al /bin /bin/bsh /bin/sh /lib /u /unix Here is an example output:

lrwxrwxrwx 1 root sys 8 Aug 5 2001 /bin -> /usr/bin -r-xr-xr-x 3 bin bin 256224 Jun 4 2001 /bin/bsh

-r-xr-xr-x 3 bin bin 256224 Jun 4 2001 /bin/sh

lrwxrwxrwx 1 root sys 8 Aug 5 2001 /lib -> /usr/lib lrwxrwxrwx 1 root sys 5 Aug 5 2001 /u -> /home

lrwxrwxrwx 1 root sys 18 Aug 5 2001 /unix -> /usr/lib/boot/unix

If any of these files are missing, the problem may be a missing symbolic link.

Use the commands from the following list that correspond to the missing links:

# ln -s /usr/bin /bin

# ln -s /usr/lib/boot/unix /unix

# ln -s /usr/lib /lib

# ln -s /home /u

10.Use the following command to make sure that fsck and rc.boot are not missing or corrupt:

# ls -l /etc/fsck /sbin/rc.boot Example output:

lrwxrwxrwx 1 root system 14 Aug 5 2001 /etc/fsck -> /usr/sbin/fsck -rwxrwxr-- 1 root system 33760 Aug 30 2001

11.Make sure the /etc/inittab file is for AIX V4 or AIX 5L V5.1. For both versions, the line that begins with brc is:

brc::sysinit:/sbin/rc.boot 3 >/dev/console 2>&1 See Example 3-1 for a sample of the /etc/inittab file.

Example 3-1 Sample /etc/inittab file init:2:initdefault:

brc::sysinit:/sbin/rc.boot 3 >/dev/console 2>&1 # Phase 3 of system boot powerfail::powerfail:/etc/rc.powerfail 2>&1 | alog -tboot > /dev/console # Power Failure Detection

load64bit:2:wait:/etc/methods/cfg64 >/dev/console 2>&1 # Enable 64-bit execs rc:23456789:wait:/etc/rc 2>&1 | alog -tboot > /dev/console # Multi-User checks fbcheck:23456789:wait:/usr/sbin/fbcheck 2>&1 | alog -tboot > /dev/console # run /etc/firstboot

srcmstr:23456789:respawn:/usr/sbin/srcmstr # System Resource Controller rctcpip:23456789:wait:/etc/rc.tcpip > /dev/console 2>&1 # Start TCP/IP daemons

rcnfs:23456789:wait:/etc/rc.nfs > /dev/console 2>&1 # Start NFS Daemons cron:23456789:respawn:/usr/sbin/cron

piobe:2:wait:/usr/lib/lpd/pio/etc/pioinit >/dev/null 2>&1 # pb cleanup qdaemon:23456789:wait:/usr/bin/startsrc -sqdaemon

writesrv:23456789:wait:/usr/bin/startsrc -swritesrv uprintfd:23456789:respawn:/usr/sbin/uprintfd

shdaemon:2:off:/usr/sbin/shdaemon >/dev/console 2>&1 # High availability daemon l2:2:wait:/etc/rc.d/rc 2

l3:3:wait:/etc/rc.d/rc 3 l4:4:wait:/etc/rc.d/rc 4 l5:5:wait:/etc/rc.d/rc 5 l6:6:wait:/etc/rc.d/rc 6 l7:7:wait:/etc/rc.d/rc 7 l8:8:wait:/etc/rc.d/rc 8 l9:9:wait:/etc/rc.d/rc 9

logsymp:2:once:/usr/lib/ras/logsymptom # for system dumps pmd:2:wait:/usr/bin/pmd > /dev/console 2>&1 # Start PM daemon

itess:23456789:once:/usr/IMNSearch/bin/itess -start search >/dev/null 2>&1 diagd:2:once:/usr/lpp/diagnostics/bin/diagd >/dev/console 2>&1

ctrmc:2:once:/usr/bin/startsrc -s ctrmc > /dev/console 2>&1

12.If you have not found any obvious problems, try substituting ksh for bsh with the following commands (the first command saves your bsh before you copy over it):

# cp /bin/bsh /bin/bsh.orig

# cp /bin/ksh /bin/bsh

If you can then reboot successfully, you know that one of the profiles was causing problems for bsh. Check the profiles again by running the following:

# /bin/bsh.orig /.profile

# /bin/bsh.orig /etc/profile

# /bin/bsh.orig /etc/environment

If you receive errors with any of these commands, you know there is a command in that profile that bsh cannot handle.

If you followed all of the preceding steps, and the system still stops at an LED 553 during a reboot in Normal mode, you may want to ask for further system recovery assistance from your AIX support personnel.

For reasons of time, and for the integrity of your AIX operating system, the best alternative at this point may be to reinstall AIX.