This chapter provides information on identifying the cause of problems with The MESH control network.
The MESH Control Network Addresses
All I/A Series equipment IP addresses for The MESH control network are 151.128.81.x (where x is a number between 1 and 254 that identifies the specific Ethernet switch). IP addresses assigned to the primary port of controllers (FCP270, ZCP270), FCM100Ets, FCM100Es, and worksta-tions are 151.128.y.z (where y is a number between 152 and 191, and z is a number between 1 and 254). The IP addresses for the alternate ports of FCM100Ets, FCM100Es or workstations are 151.128.w.z (where w is derived from y above as w = y - 128, and z is the same value as above).
The IP addresses for the primary I/O controller ports of ZCP270 are 151.128.v.z (where v is derived from y above as v = y + 64, and z is the same value as above). The IP addresses for the alternate Input/Output Controller (IOC) ports of ZCP270 are 151.128.u.z (where u is derived from y above as u = y - 64, and z is the same value as above). See Table 4-1.
Fault tolerant FCP270s and ZCP270s use the same IP addresses for both modules of the fault tol-erant pair. Additionally, the ZCP270 requires a third and fourth IP address for its Input/Output Controller (IOC) ports. These also are derived and assigned according to Table 4-1.
IP addresses for all devices on The MESH control network (workstations, controllers, managed switches, FCM100Ets and FCM100Es) are assigned by the system configurator (SysDef or IACC). For information on using the system configurator, refer to:
♦ I/A Series System Definition: A Step-by-Step Procedure(B0193WQ) or
♦ I/A Series Configuration Component (IACC) User’s Guide (B0400BP).
Table 4-1. IP Address Assignments
I/A Series Release Port IP Address
v8.0 or later Network Monitoring Devices 151.128.82.1 through 151.128.82.254 v8.0 or later Primary Port 151.128.152.1 through 151.128.191.254 v8.0 to v8.1.x Alternate Port1
1. At v8.2, the intermediate driver was changed so that the Alternate Port IP is never used.
151.128.24.1 through 151.128.63.254 v8.0 or later Primary IOC Port 151.128.216.1 through 151.128.255.254 v8.0 or later Alternate IOC Port 151.128.88.1 through 151.128.127.254 v8.0 or later Switches 151.128.81.1 through 151.128.81.254
Pre-v8.0 Single Node 151.128.8.65 through 151.128.8.126
Pre-v8.0 Multi-Node 151.128.16.65 through 151.128.23.254
Pre-v8.0 Switches 151.128.79.1 through 151.128.79.254 or
151.128.80.1 through 151.128.80.254
B0700AZ– RevL 4.Maintenance
Once the configurator has assigned the primary IP address, each Ethernet switch must be config-ured with its assigned IP address using the Command Line Interface (CLI) Entries or I/A Series Switch Configurator Application Software (discussed in the switch configuration section of the documentation included with your Invensys qualified Ethernet switches). One IP address is assigned to each non-Chassis Ethernet switch in The MESH control network. For the N1 Chassis switch (P0973AR), the chassis is assigned one IP address. For the N3 Chassis switch (P0973AS), the chassis is assigned one IP address for each of its three blades. For the N7 Chassis switch (P0972YE), the chassis is assigned one IP address for each of its seven blades.
The system configurator (SysDef or IACC) assigns a MAC (Media Access Control) address for each FCP270, ZCP270, FCM100Et, and FCM100E on the network. The primary port MAC addresses are assigned from the range, 00006CC00000 to 00006CC03FFF. The alternate port MAC addresses are derived from the primary port MAC addresses by “ORing” the primary net- work MAC address with 000000004000, yielding a range of 00006CC04000 to
00006CC07FFF. The MAC addresses for the primary I/O controller ports of ZCP270 are derived from the primary port MAC addresses by “ORing” the primary network MAC address with 000000008000, yielding a range of 00006CC08000 to 00006CC0BFFF. The MAC addresses for the alternate I/O controller ports of ZCP270 are derived from the primary port MAC addresses by “ORing” the primary network MAC address with 00000000C000, yielding a range of
00006CC0C000 to 00006CC0FFFF.
Workstations, FCP270, ZCP270, FCM100Et, FCM100E and FBMs are assigned letterbugs to identify the module by the system configurator. FBMs are not assigned IP or MAC addresses. Let-terbugs and IP addresses can be changed using the system configurator
General Troubleshooting Guidelines
When a problem occurs with The MESH control network, it is usually best to take an organized approach to diagnosing the cause. A random or scattered approach generally takes longer and can make it more difficult to track down the cause of the problem by introducing additional
unknowns. The following sections will help to determine the cause of a network problem.
Characterize the Problem
The initial step in diagnosing network problems is to understand what kind of problem exists.
♦ Has the problem always existed or has it just started?
New problems are sometimes caused by a change in The MESH control network figuration. Check to see whether there have been recent changes to The MESH con-trol hardware, software, or configuration files.
♦ Is the problem constant or intermittent?
Constant problems are often the result of a component failure or a change to The MESH control network configuration. Intermittent problems can be caused by envi-ronmental factors such as excess heat, electrical noise, poor contacts, or high attenua-tion.
♦ What kind of error indication is occurring?
Device communication failure - inability to communicate with one or more devices
Data Loss - incorrect data is arriving at one or more devices. The data path is intact, but the data is being corrupted along the way. This can be caused by failing network devices, environmental factors, or exceeding transmission distance limits.
Error message - software has detected a problem. This can be caused by any number of things. The text of the error message indicates what kind of problem has been detected.
Determine Which Devices are Affected
Identifying the devices that are affected helps to determine where the cause of the problem lies.
For example, if all the unreachable devices are connected to the same Ethernet switch, then you should probably start troubleshooting at that switch.
When troubleshooting a Security Enhanced Configuration with LDP deployed, it is highly rec-ommended that you refer to the section “Loop Detection Monitoring and Maintenance” inThe MESH Control Network Operation, and Switch Installation and Configuration Guide(B0700CA) before continuing.
Follow the steps below to identify which device or devices are affected.
1. Are the problems limited to specific devices (stations, switches), or are they general in nature?
2. Are the problems constant or are they intermittent?
3. Can the devices be reached using theping command?
4. Does theping -s command report that packets are being dropped?
5. What does System Management Displays indicate? Refer to “System Management Displays” on page 74.
6. Use a network topology map to determine the physical location of the affected devices.
Troubleshoot the Affected Devices
1. Check activity and status indicators on the affected devices.
2. Verify that the devices are getting power (check Status LEDs).
3. Check that the devices are correctly cabled and that the cables are not damaged (swap a suspect cable with a known good cable).
4. Verify that the limit on distance between devices has not been exceeded. Distance is dependent on transmission mode and cable type.
5. If fiber optic cable is being used, verify that the optical budget (total signal loss - dB), bandwidth, and dispersion are within specification.
6. Verify that traffic rates to the affected devices are not excessive.
7. Check System Management error counters. Refer toSystem Management Displays (B0193JC).
8. Verify that the devices are properly configured. Also check that the adjacent device in the network is correctly configured as well.
B0700AZ– RevL 4.Maintenance
NOTE
Before removing a switch from the network, be sure to record its existing configura-tion. This information is necessary when the replacement switch is configured.
NOTE
Make sure a replacement switch is correctly configured before adding it to the net- work. Any uplink ports should be installed before the switch is configured. Refer to
the documentation included with your Invensys qualified Ethernet switches for more information.
System Management Displays
The I/A Series System Management Display Handler (SMDH) obtains current and historical information about the system, and displays this information in System Management displays.
SMDH uses SNMP to access the switches to obtain status information from each switch. With regard to The MESH control network and associated Ethernet switches, SMDH provides the fol-lowing displays:
♦ A Switched Network View of all the configured switches
♦ A Switch Domain display that shows all The MESH control network stations in the selected Switch’s domain.
♦ A Switch Ports Display which shows each port of the switch that was selected on the Domains display
♦ Detailed equipment information (EQUIP INFO) and equipment change (EQUIP CHG) displays for each port of the selected Ethernet switch.
Accessing SMDH Switch Network Displays
System Management Displays are accessed from the FoxView™ application.
To access System Management on Windows® based systems or Solaris® based systems with the FoxView application, refer toSystem Management Displays (B0193JC).
The initial System Management Display, System Monitor Domains, appears. From this display you can select a system monitor and navigate through the Domain Display(s).
All switches are shown in the system monitor domain as stations designated by their letterbug. To access displays for The MESH control network switches, select (click-on) the menu bar soft keys indicated in Figure 4-1.
Figure 4-1. Accessing SMDH Switched Network Displays
B0700AZ– RevL 4.Maintenance
Switched Network Display
A typical Switched Network display is shown in Figure 4-2. The display is accessed from the Sys-tem Monitor Domains display by clicking on the
It displays the letterbug “system name” of every switch in the network. Note that each card (blade) in an E7 Chassis switch is assigned a letterbug, where the N7 Chassis switch is only assigned one letterbug. Media converters and extenders are not assigned letterbugs.
From the Switched Network Display you can select a switch (click on the letterbug of a particular switch of interest) and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or NEXT LEVEL. Clicking NEXT LEVEL brings up the Switch Domain Display.
Figure 4-2. SMDH Switched Network Display SW NTWRK soft key.
Switch Equipment Change Display
A typical Switch Equipment Change display is shown in Figure 4-3. The display is accessed from the Switched Network display or the Switch Domain display.
The Switch Equipment Change display enables/disables station (switch) alarms and reports from being propagated to the I/A Series system. Table 4-2 describes only the available text fields in the order that they appear on the display pages, from left column to right column.
Figure 4-3. Switch Equipment Change Display
Table 4-2. Switch Equipment Change Display Actions
Action Description
Enables device status to affect the overall system status.
Inhibits device status to affect the overall system status.
Enables reports between the device and the I/A Series System Man-agement.
Disables reports between the device and the I/A Series System Man-agement.
ENABLE STATION ALARMING
INHIBIT STATION ALARMING
ENABLE
ALL REPORTS DISABLE ALL REPORTS
B0700AZ– RevL 4.Maintenance
Switch Equipment Information Display
A typical Switch Equipment Information display is shown in Figure 4-4. The display is accessed from the Switched Network display or the Switch Domain display. It shows the name, type, and status of the switch.
Table 4-3 describes only the available text fields in the order that they appear on the display pages, from left column to right column.
Figure 4-4. Switch Equipment Information Display
Table 4-3. Switch Equipment Information Display Fields
Field Description
Name (letterbug) of the selected station.
On-line or Off-line (default) is displayed.
Acknowledged (default) or Not Acknowledged is displayed. If the value changes to Not Acknowledged to indicate this transition, and remains until you acknowledge the switch failure.
Acknowledged (default) or Not Acknowledged is displayed. If any of the switch ports fail and become unacknowledged, Not Acknowledged is displayed.
The MAC Address of the station.
The number of ports for a particular switch: Sw w/16 ports, Sw w/24 ports, Sw w/48 ports. For the N1, N3, N7 Chassis switches, displays
STATION NAME RUN MODE
FAIL ACK STATE FAIL
STATE value changes from Not Failed to Failed, theFAIL ACK STATE
FAIL DEV ACK
MAC ADDRESS TYPE
FAIL STATE Fail or Not Failed is displayed.
ALARMING STATE Alarming State indicates whether alarming is enabled or inhibited for the switch. When alarming is inhibited, the System Monitor continues to indicate overall system and network health (a green
Yes is displayed if one or more switch ports has failed; otherwise, No is displayed.
MT REPORT STATE Master Timekeeper Report State. GPS Not Configured is always dis-played for the switch equipment information display.
Table 4-3. Switch Equipment Information Display Fields (Continued)
Field Description
SYS BAR).
FAIL DEV ATT
B0700AZ– RevL 4.Maintenance
Switch Configuration Information Display
A typical Switch Configuration Information display is shown in Figure 4-5. The display is accessed from the Domains display.
Table 4-4 describes the available text fields in the order that they appear on the display pages, from left column to right column.
Figure 4-5. Switch Configuration Information Display
Table 4-4. Switch Configuration Information Display Fields
Field Description
Name (letterbug) of selected station.
System Monitor name for the selected station.
IP Address for the selected station.
SW CONNECTION Lists all of the switches to which this switch’s ports are connected such as; SW001B, SW003A, SW004B.
The number of ports for a particular switch: Sw w/16 ports, Sw w/24 ports, Sw w/48 ports, up to a Sw w/672 ports.
SMON AP NAME The name (letterbug) of workstation that is hosting SMON.
BOOT HOST Name (letterbug) of boot host for the switch.
STATION NAME SMON NAME IP ADDRESS
TYPE
Switch Domain Display
The Switch Domain display for a switch is shown in Figure 4-6. The display is similar for all switches.
From the Switch Domain Display you can select a switch (click on the letterbug of the switch) and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or NEXT LEVEL. The EQUIP CHG, EQUIP INFO, CONFIG INFO displays are the same as those that can be selected from the Switched Network Display. Clicking NEXT LEVEL brings up the Switch Switch Ports Display.
Figure 4-6. Switch Domain Display
The SWITCH DOMAIN DISPLAY indicates the status of The MESH control network Ethernet cables to/from the modules connected to the switch. If a fault in cable A or cable B is detected, the faulty Ethernet cable is identified by the mnemonic RA (Receive A), RB (Receive B), or RAB (Receive A and Receive B) appearing next to the receive cable as shown below:
R
B0700AZ– RevL 4.Maintenance
Switch Ports Display
A typical Switch Ports Display for a 16-port switch is shown in Figure 4-7. The Switch Ports Dis-play, displays a maximum of 30-ports on one screen. Paging is enabled so that more than 30-ports can be shown on the display.
The carrot < next to the port number in Figure 4-7 indicates that the port alarming has been inhibited (see “Switch Port Equipment Change Display” on page 83). All unused ports must be inhibited, disabling switch alarming for that port. This stops false alarms from propagating upward in the I/A Series System Management.
The asterisk “*” adjacent to the port number in Figure 4-7 indicates that the port has failed “Lost Link” and should be investigated to determine the cause of the port failure. Refer to“General Troubleshooting Guidelines” on page 72 to aid in the isolation of the failure.
Figure 4-7. Switch Ports Display - Typical
Switch Port Equipment Change Display
The Equipment Change display for a switch is shown in Figure 4-8. The display is similar for all switch ports. Table 4-5 describes the available text fields in the order that they appear on the dis-play pages, from left column to right column.
Figure 4-8. Switch Equipment Change Display
Table 4-5. Switch Equipment Change Actions
Action Description
Enables switch alarms to propagate upward in the I/A Series System Management.
Inhibits switch alarms from propagating upward in the I/A Series System Management. All non-used Ethernet ports must be inhibited for proper device monitoring.
ENABLE DEVICE ALARMING
INHIBIT DEVICE ALARMING
B0700AZ– RevL 4.Maintenance
Switch Port Equipment Information Display
An Equipment Information display for a port of an Ethernet switch is shown inFigure 4-9. The display is similar for all switches. Table 4-6 describes the available text fields in the order that they appear on the display pages, from left column to right column.
Figure 4-9. Switch Port Equipment Information Display - Typical
Table 4-6. Switch Port Equipment Information Display Fields
Field Description
The port number of the switch displayed in SMDH as P01 through P378.
On-line or Off-line (default) is displayed.
Acknowledged (default) or Not Acknowledged is displayed. If the
and remains until you acknowledge the port failure.
Yes is displayed if the device has a non-fatal error condition; otherwise, No is displayed.
NUMBER The port number 1 through 64.
CURRENT STATE Port running or port failed.
TYPE The switch port or the uplink of the switch.
NAME
RUN MODE
FAIL ACK STATE
DEVICE STATE value changes from Not Failed to Failed, theFAIL ACK STATE value changes to Not Acknowledged to indicate this transition,
WARNING STATE
Indicators
Ethernet Switches
For the location and meaning of the indicators on a particular Ethernet switch, refer to the docu-mentation provided with the switch.
Media Converter
For the location and meaning of the indicators on a particular media converter, refer to the docu-mentation provided with the device.
Fiber Optic Cable Handling and Cleaning
Fiber optics communication relies on a clear path for its signals. Make every effort to install splices, connectors, and terminations as clean as possible, thus reducing their effects on optical data transmission.
Handling Fiber Optic Cable
Consult the cable specifications for the cable you are installing. Mishandling the cable may cause damage that can alter its transmission characteristics requiring replacement of the cable. When handling fiber optic cabling:
Alarming State indicates whether alarming is enabled or inhibited for the device. When alarming is inhibited, the System Monitor continues to indicate overall system and network health (a green
equipment is Failed or Off-line.
OK or Fail.
Port Enabled or Port Disabled.
Interface state indicates the communications state of the port. Ethernet-csmacd or software Loopback are normally displayed for The MESH control network. Other states that can be displayed are: other,
regular1822, hdh1822, ddn-25, rfc877-x25, iso88023-csmacd,
iso88024-tokenbus, iso88025-tokenRing, iso88026-man, starLan, pro-teon-10 Mb, proteon-80Mb, hyperchannel, fddi, lapb, sdlc, dsl, el, basic ISDN, proppointToPointSerial, ppp, eon, ethernet-3Mb, nsip, slip, ultra, ds3, sip, or fram-relay. These other states can be shown depending on the type of switch and communications employed.
Table 4-6. Switch Port Equipment Information Display Fields (Continued)
Field Description
ALARMING STATE
SYS BAR) while
COMM STATE ADMIN STATE INTERFACE STATE
B0700AZ– RevL 4.Maintenance
WARNING
!
Never look directly into the end of a fiber cable or bulkhead adapter. Eye damage may result. Laser light can damage your eyes. Laser light is invisible. Viewing it directly does not cause pain. The iris of the eye will not close involuntarily as when viewing a bright light. Consequently, serious damage to the retina of the eye is pos-sible. Never look into the end of a fiber or at a connector, or a connector in an inspection microscope which may have a laser coupled to it. Should accidental eye exposure to laser light be suspected, arrange for an eye examination immediately.
♦ Turn off power to the equipment before cleaning or looking at fiber cable ends.
♦ Always use dust covers on the end of the fiber cable connector (see Figure 4-10, Figure 4-11 and Figure 4-12) or any bulkhead (including unused bulkheads)
♦ Do not pull or kink the cable as the glass strand in the middle might become damaged or broken
♦ Do not walk-on, step-on, or crush the cable as the glass strand in the middle might
♦ Do not walk-on, step-on, or crush the cable as the glass strand in the middle might