Cisco IOS XR Diagnostics

(1)

Cisco IOS XR Diagnostics

Cisco IOS XR online diagnostics allow you to test and verify hardware functionality while connected to a live network. Health-monitoring and scheduled diagnostics help ensure system high availability (HA). If a problem is detected, online diagnostics help isolate the location of the problem allowing you to identify and replace the hardware. Online diagnostics are also used for system acceptance of new hardware.

The online diagnostics contain tests that check different hardware components and verify the data path and control signals. Nondisruptive online diagnostic tests can be scheduled or run on-demand.

On-demand diagnostics run from the command-line interface (CLI); scheduled diagnostics run at user-designated intervals or specified times when the system is connected to a live network.

Online diagnostics is one of the requirements for high availability (HA). High availability is a set of quality standards that seek to limit the impact of equipment failures on the network. A key part of high availability is detecting hardware failures and taking corrective action while the system continues to run in a live network. Online diagnostics detect hardware failures and provide feedback to high availability software components.

Note Online diagnostics is supported on all cards except Shared Port Adapter (SPA) cards. Offline diagnostics is supported on all nodes. For line card service processor (SP), SPA and physical layer interface module (PLIM), offline diagnostics must be run from line card CPU node. Each CPU node has its own diagnostic process, or test execution engine, but is part of the overall diagnostic system.

Note Diagnostic messages are logged in syslog. See the Implementing Logging Services on Cisco IOS XR

Software module of Cisco IOS XR System Monitoring Configuration Guide or use the show logging | inc

diag command to display the contents of the logging buffer.

Integrated Field Diagnostics (IFDs) allow you to load and unload offline diagnostic images. Loading a diagnostic image places the specified location out of service. When an offline diagnostic image is loaded, the diagnostic start command is used to start the test, and the show diagnostic content and show diagnostic results commands are used to show the test list and results. See the Cisco IOS XR Interface and Hardware

Components Command Reference for diagnostic command information.

The integrated field diagnostics detect problems in the following areas: • Hardware components

• Connectors (loose connectors, bent pins, and so forth) • Solder joints

(2)

Cisco IOS XR Diagnostics

Note The Cisco IOS XR Diagnostics Package must be installed to access the Cisco IOS XR diagnostics. See

Cisco IOS XR Getting Started Guide for information on installing packages.

Note For more information about diagnostics on the Cisco IOS XR software and complete descriptions of diagnostics commands listed in this module, see the “Related Documents” of this module. To locate documentation for other commands that might appear during the execution of a configuration task, search online in the Cisco IOS XR software master command index.

Feature History for Cisco IOS XR Diagnostics

The following information is provided: • Diagnostics Framework, page 3

• Diagnostics Operations, page 3

• Available Online Diagnostic Tests, page 4

• Running Cisco IOS XR Online Diagnostics, page 6

• Running Integrated Field Diagnostics, page 10

• Displaying Diagnostic Test Results, page 15

• Displaying the Diagnostic Schedule, page 15

• Examples for Running Cisco IOS XR Diagnostics, page 16

• Additional References, page 22

Release Modification

Release 3.3.0 This feature was introduced on the Cisco CRS-1.

Release 3.4.0 Configuring health-monitoring diagnostics, including support for monitor syslog, monitor intervals, and the failure count thresholds of test, was added.

The show diagnostic result and show diagnostic content command output was modified to include diagnostic monitoring support.

The show diagnostic content command output for nodes in the FDIAG RUNNING state has expanded to provide more control over execution of offline test suites.

The following online diagnostics tests were added: • Control Ethernet Inactive Link Test

(3)

Diagnostics Framework

Diagnostics Framework

Cisco IOS XR online diagnostics defines a common framework for diagnostic operations across Cisco platforms running Cisco IOS XR software. Figure 1 illustrates the diagnostics framework and how it interacts with platform dependant diagnostics operations.

The command-line interface (CLI) categorizes diagnostics as runtime diagnostics. Cisco IOS XR online diagnostics also provides for easy addition of platform specific tests that contribute in reducing the Mean Time To Repair (MTTR).

The Cisco IOS XR online diagnostics framework specifies platform-independent fault detection architecture for centralized and distributed systems. This framework includes common diagnostics CLI and platform-independent fault detection procedures for runtime diagnostics.

Figure 1 Diagnostic Framework

Diagnostics Operations

Cisco IOS XR online diagnostics are implemented using tests to check the health of hardware components and to verify proper operation of the system data and control paths. As illustrated in

Figure 1, tests can be categorized into runtime and offline diagnostic tests.

• Runtime Diagnostics—Defects can be diagnosed during system operation or runtime. A series of diagnostic checks can be enabled to determine the condition of an online system. Care must be taken to distinguish between disruptive and non-disruptive diagnostics tests. While non-disruptive tests occur in the background and do not affect the data or control plane of a system, disruptive tests will affect live packet flows and should be scheduled during special maintenance windows. The impact of disruptive tests can be minimal or significant. Disruptive tests can take varying amounts of time to complete.

Runtime diagnostic checks can be run on demand, can be scheduled to run at a specific time, or can run in the background:

On-demand Scheduled Runtime diagnostics

Online diagnostics layer

Generic online diagnostics subsystem framework

Health monitoring

Platform specific diagnostic

Disruptive and non-disruptive online diagnostics Non-disruptive health monitoring Hardware HW layer Detect bad HW Provide generic diagnostics & health monitoring framework

(4)

Cisco IOS XR Diagnostics Available Online Diagnostic Tests

– Online diagnostics—Online diagnostics tests are triggered by entering a diagnostic start command. You can specify how many times a test runs and whether to continue the test on failure detection. Online diagnostics is useful as a troubleshooting tool that allows you to verify hardware functionality when a hardware fault is suspected. Note that online diagnostics will not cause bad hardware to reset or power down on the Cisco CRS-1. Syslog messages warn about bad hardware and you need to check the diagnostics results to understand if the tests passed or failed and take further action.

– Scheduled diagnostics—Scheduled diagnostic tests are scheduled to run either at one specific time or periodically. This is especially useful when scheduling disruptive tests during

maintenance windows. When failures are detected, appropriate syslog messages are displayed and diagnostic results are saved. Scheduled diagnostics do not cause bad hardware to reset or power down on the Cisco CRS-1.

– Health-monitoring diagnostics—Health-monitoring diagnostic tests are nondisruptive and run in the background while the system is in operation. The role of health-monitoring diagnostics is to proactively detect hardware failures in the live network environment and inform

appropriate entities of a failure. It is up to you to determine the number of health-monitoring checks and the interval at which to run health-monitoring checks. The interval can be as granular as 50ms. By default, health-monitoring tests include data path and control path verification as well as proper hardware registers functionality.

• Offline Diagnostics—Offline diagnostic tests run field diagnostics on a node.

Available Online Diagnostic Tests

The following online diagnostic tests are supported:

• Control Ethernet Ping Test—A nondisruptive test that "pings" each control ethernet node in a chassis from the node where the test is started. The test sends a ping to one node at a time, waiting for a response or until the maximum timeout of 2 seconds is reached, before proceeding to send a ping to the next node. The returned ping response is verified by comparing it byte by byte with the sent ping. Pings are sent only to active nodes within the same chassis as the node from which the test is run. Only one ping is sent per node. Each ping has a 100 byte payload. The test result is PASS if all nodes return the ping and the response matches the sent ping.

• Fabric Ping Test—A nondisruptive test that "pings" each fabric node in a chassis from the node where the test is started. The test sends a ping to one node at a time, waiting for a response or until the maximum timeout of 2 seconds is reached, before proceeding to send a ping to the next node. The returned ping response is verified by comparing it byte by byte with the sent ping. Pings are sent only to active nodes within the same chassis as the node from which the test is run. Due to the undeterministic path of traffic in the fabric, 72 pings are sent per node to maximize coverage. Each ping has a 1-kilobyte payload. The test result is PASS if all nodes return all the pings and all the responses match the sent pings. Due to the 2 second wait timeout for a lost ping, a node that is unreachable or intermittently working impacts the total run time for the test. Therefore, in a worst case scenario where a node has lost all fabric connectivity, the test time can be increased by 2.5 minutes for that node. The total test time depends on how many active nodes there are in the tested rack and how many of the nodes have failing fabric connections.

(5)

Available Online Diagnostic Tests

– One response that travels along the inactive CE return link, which verifies the inactive CE link from test target to standby RP

– One response that travels along the active CE link through the active RP CPU to the standby RP, which verifies the external inactive CE link from the standby RP to the test target

– One that travels along the active CE link to the standby RP, which verifies the internal test target CE path

Each returned response is verified by comparing it byte-by-byte with the sent ping. When either all three responses from the test target have returned, or a timeout of 2 seconds has expired, the test repeats this procedure for the next node in the rack until all nodes connected to inactive CE links are tested. The test result is PASS if all nodes return all the pings and all the responses match the sent pings. This test can be used to "qualify" the CE connectivity of the standby RP prior to a switchover. • Self-Ping over Fabric Test—A nondisruptive test that lets a node "ping" itself over the fabric. The test sends a fabric ping to the node itself, waiting for a response or until the maximum timeout of 2 seconds is reached. The returned ping response is verified by comparing it byte by byte with the sent ping. Each ping has a 100-byte payload. This single ping is repeated 100 times with an interval of 300 ms in between each ping. The test result is PASS if all the pings are returned and all the responses match the sent pings. This test is equivalent to the existing health monitoring fabric ping test that runs automatically on all LC nodes. It differs in that it is available on all nodes with fabric connectivity, not only LCs, and in that its health monitoring characteristics (enabled/disabled, run interval, and so on) can be configured by the user. The normal run time for this test is 30 seconds. Due to the 2 second wait timeout for a lost ping, the test time for a node that has failing fabric connectivity may be increased by up to 3.5 minutes.

• RommonRevision Test—A nondisruptive test that verifies that a node is running the minimally acceptable version of ROM Monitor (ROMMON). When a node is rebooted, its current version of ROMMON is retrieved and saved in a shared memory space. This shared memory space is queried for the running version of ROMMON and that version is compared to the minimally acceptable version of ROMMON. If the running version is not greater than or equal the minimal version, the test fails.

• Fabric Diagnosis Test—A non-disruptive, fault isolation test that "pings" each fabric node in a chassis from the standby RP node. The test steers ping test packets through different fabric planes, aggregates ping (pass or fail) results with fabric plane information, analyzes the results, and points out the most logical point of failure (if any) in the chassis. The test can only be executed from the standby RP on demand. Executing this test in a Cisco CRS-1 Carrier Routing System Multishelf System may help determine which fabric stage (s1, s2, or s3) is the most logical point of failure in the system. This test must be run in each LC rack standby RP in the system. For example, if the test reports failures on multiple LC racks, and the failure information points to the same fabric plane, then the most likely point of failure is the S2 stage, which is the card in fabric chassis of the system. We highly recommend that you only run this test when the system experiences fabric issues and needs to isolate the fault.

(6)

Cisco IOS XR Diagnostics Running Cisco IOS XR Online Diagnostics

Running Cisco IOS XR Online Diagnostics

The following options are available for running online diagnostics:

• Running On-Demand Online Diagnostics, page 6

• Scheduling Online Diagnostics, page 7

• Configuring Health-Monitoring Diagnostics, page 9

Note For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface

and Hardware Components Command Reference.

Running On-Demand Online Diagnostics

To start and stop on-demand online diagnostics, perform the following procedure. A message is displayed when the diagnostic tests are completed.

Note The diagnostic stop command is used for stopping a test before it has completed. For example, you can stop a test that takes longer than desired. See Cisco IOS XR Interface and Hardware Components

Command Reference for syntax information.

Prerequisites

The Cisco IOS XR Diagnostics Package must be installed before starting this procedure. See

SUMMARY STEPS

1. admin

2. show diagnostic content location node-id

(7)

Running Cisco IOS XR Online Diagnostics

DETAILED STEPS

What to Do Next

Proceed to the “Displaying Diagnostic Test Results” section on page 15 to display the results from the on-demand diagnostic test.

Scheduling Online Diagnostics

You can schedule online diagnostics to run at a designated time of day or on a daily or weekly basis for a specific node. You can schedule tests to run only once or to repeat at an interval. Use the no form of the diagnostic schedule command to remove the scheduling.

To schedule online diagnostics, perform the following procedure.

Prerequisites

SUMMARY STEPS

1. admin

2. show diagnostic content location node-id 3. configure

4. diagnostic schedule location node-id test {id | all | basic | non-disruptive} {daily | on month day

year | weekly day-of-week} hour:minute

5. end

6. show diagnostic schedule location node-id

Command or Action Purpose

Step 1 admin

Example:

RP/0/RP0/CPU0:router# admin

Enters administration EXEC mode.

Step 2 show diagnostic content location node-id

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0

Displays the available tests and their attributes, including which commands are in the nondisruptive category.

Step 3 diagnostic start location node-id test {id |

all | basic | non-disruptive}

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test 1

(8)

Cisco IOS XR Diagnostics Running Cisco IOS XR Online Diagnostics

DETAILED STEPS

Example:

Step 2 show diagnostic content location node-id

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0

Displays the available tests and their attributes, including which commands are in the nondisruptive category.

Step 3 configure

Example:

RP/0/RP0/CPU0:router(admin)# configure

Enters administration configuration mode.

Step 4 diagnostic schedule location node-id test {id |

all | basic | non-disruptive} {daily | on month

day year | weekly day-of-week} hour:minute

Example:

RP/0/RP0/CPU0:router(admin-config)# diagnostic schedule location 0/RP1/CPU0 test 1 daily 12:42

Schedules on-demand diagnostic tests for a specific location and date and time.

Step 5 end

Example:

RP/0/RP0/CPU0:router(admin-config)# end

Saves configuration changes.

When you issue the end command, the system prompts you to commit changes:

Uncommitted changes found, commit them before exiting(yes/no/cancel)?

[cancel]:

– Entering yes saves configuration changes to the running configuration file, exits the configuration session, and returns the router to EXEC mode. – Entering no exits the configuration session and

returns the router to EXEC mode without committing the configuration changes.

– Entering cancel leaves the router in the current configuration session without exiting or committing the configuration changes. Step 6 show diagnostic schedule location node-id

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/RP1/CPU0

(9)

Running Cisco IOS XR Online Diagnostics

What to Do Next

Proceed to the “Displaying Diagnostic Test Results” section on page 15 to display the results from the scheduled diagnostic test.

Proceed to the “Displaying the Diagnostic Schedule” section on page 15 to display the scheduled diagnostic tasks.

Configuring Health-Monitoring Diagnostics

To configure health-monitoring diagnostic testing on specified cards while the router is connected to a live network, perform the following procedure. You can configure the:

• Execution interval for each health-monitoring test • Failure threshold for each health-monitoring test • Generation of system messages upon test failure • Location where each health-monitoring test is run

Prerequisites

SUMMARY STEPS

1. admin 2. configure

3. diagnostic monitor syslog

4. diagnostic monitor location node-id test {id | test-name} [disable]

5. diagnostic monitor interval location node-id test {id | test-name} number-of-days

hour:minutes:seconds.milliseconds

6. diagnostic monitor threshold location node-id test {id | test-name} failure count failures

DETAILED STEPS

Example:

Step 2 configure

Example:

RP/0/RP0/CPU0:router(admin)# configure

(10)

Cisco IOS XR Diagnostics Running Integrated Field Diagnostics

What to Do Next

Proceed to the “Displaying Diagnostic Test Results” section on page 15 to display the results of the health-diagnostic monitoring output.

Running Integrated Field Diagnostics

To load and unload an offline diagnostics image for integrated field diagnostics, perform one of the following procedures:

• Autostart Integrated Field Diagnostics, page 11

• Manually Executing Integrated Field Diagnostics, page 13

Loading an image places the specified card out of service. Unloading the image returns the specified card to service.

Caution To run offline diagnostics successfully, ensure there are no external fiber cables connected to the line card PLIM ports and that all PLIM ports are populated with optics or small form-factor pluggable (SFP) optic modules. See the Cisco IOS XR hardware documentation on Cisco.com for information on PLIM ports.

Step 3 diagnostic monitor syslog

Example:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor syslog

(Optional) Enables the generation of a syslog message when any health-monitoring test fails.

Step 4 diagnostic monitor location node-id test {id |

test-name} [disable]

Example:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1

Configures the health-monitoring diagnostic testing for a specified location.

Step 5 diagnostic monitor interval location node-id

test {id | test-name} number-of-days

hour:minutes:seconds.milliseconds

Example:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor interval location 0/1/cpu0 test 1 12:55:25.100

Configures the health-monitoring diagnostic testing for a specified interval for a specified location.

Step 6 diagnostic monitor threshold location node-id

test {id | test-name} failure count failures

Example:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1 failure count 25

Configures the health-monitoring diagnostic testing failure threshold.

(11)

Running Integrated Field Diagnostics

Prerequisites

Autostart Integrated Field Diagnostics

When enabled, the autostart feature for Integrated Field Diagnostics automatically starts running the tests when the image is loaded. A message is displayed when the diagnostic tests are completed. The diagnostic start command is not required. When offline diagnostics are loaded with the diagnostic load location node-id command, the specified node remains in the "FDIAG RUNNING" state until

diagnostics are unloaded using the diagnostic unload location node-id command.

The following commands can be used to unload the diagnostic test, display the test results, and display diagnostic test content when the tests are completed:

• show diagnostic result • show diagnostic content • diagnostic unload

Note The show diagnostic result location node-id command output for a specified node can be viewed until the diagnostic unload location node-id command is used. Once the test is unloaded, the test results can no longer be viewed.

While a node is in the "FDIAG RUNNING" state, tests are run in response to the optional autostart keyword of the diagnostic load location node-id command or the diagnostic start location node-id command. When an individual test finishes, a message is printed and results are updated. The results can be read using the show diagnostic result location node-id command. When a test finishes, a new diagnostic start location node-id command can be invoked, because the card remains in the "FDIAG RUNNING" state until it is explicitly unloaded using the diagnostic unload location node-id command.

SUMMARY STEPS

1. admin

2. show platform

3. diagnostic load location node-id autostart {basic | all} 4. show platform

5. show diagnostic result location node-id [test {id | all}] [detail] 6. diagnostic unload location node-id

(12)

DETAILED STEPS

Example:

Step 2 show platform

Example:

RP/0/RP0/CPU0:router(admin)# show platform

Displays the status of cards in the system. Verify that all the cards are in the IOS XR RUN state.

Step 3 diagnostic load location node-id autostart {basic | all}

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/3/CPU0 autostart all

Loads an offline diagnostic image for integrated field diagnostics, placing the specified card out of service.

Caution This command places all modules in the slot of the specified node out of service.

Note The distributed route processor (DRP) does not support the automatic running of tests when the image is loaded for CPU0 and CPU1. After the diagnostic image is loaded, use the diagnostic start location node-id test {id | all | basic |

non-disruptive} command to execute the tests. Step 4 show platform

Example:

Displays the status of cards in the system. Verify that the card state goes to the FDIAG RUNNING state. For modular services cards (MSCs), the service processor (SP) and CPU0 instance should go to the FDIAG RUNNING state. For distributed route processors (DRPs), the SP, CPU0 and CPU1 instances should all go to the FDIAG RUNNING state. Step 5 show diagnostic result location node-id [test

{id | all}] [detail]

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/3/CPU0 test 1

Displays result of tests run at the specified location. Possible results are shown as:

• Passed (.) • Failed (F) • Untested (U) Step 6 diagnostic unload location node-id

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/3/CPU0

Unloads the offline diagnostic image, placing the specified card back in service.

Example:

(13)

Running Integrated Field Diagnostics

Manually Executing Integrated Field Diagnostics

Manually executing Integrated Field Diagnostics requires you to enter the diagnostic start command to start the diagnostic testing. A message is displayed when the diagnostic tests are completed. The following commands can be used to unload the diagnostic test, display the test results, and display diagnostic test content when the tests are completed:

• diagnostic unload • show diagnostic result • show diagnostic content

Note The show diagnostic result location node-id command output for a specified node can be viewed until the diagnostic unload location node-id command is used. Once the test is unloaded, the test results can no longer be viewed.

When offline diagnostics are loaded with the diagnostic load location node-id command, the specified node remains in the "FDIAG RUNNING" state until diagnostics are unloaded using the diagnostic unload location node-id command.

While a node is in "FDIAG RUNNING" state, tests are run in response to the optional autostart keyword of the diagnostic load location node-id command or the diagnostic start location node-id command. When an individual test finishes, a message is printed and results are updated. The results can be read using the show diagnostic result location node-id command. When a test finishes, a new diagnostic start location node-id command can be invoked, because the card remains in the "FDIAG RUNNING" state until it is explicitly unloaded using the diagnostic unload location node-id command.

SUMMARY STEPS

1. admin

2. show platform

3. diagnostic load location node-id 4. show platform

5. diagnostic start location node-id test {id | all | basic | non-disruptive} 6. show diagnostic result location node-id [test {id | all}] [detail] 7. diagnostic unload location node-id

(14)

DETAILED STEPS

Example:

Displays the status of cards in the system. Verify that all the cards are in the IOS-XR RUN state.

Step 3 diagnostic load location node-id

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/3/CPU0

Loads an offline diagnostic image for integrated field diagnostics, placing the specified card out of service.

Caution This command places all modules in the slot of the specified node out of service.

Example:

Displays the status of cards in the system. Verify that the card state goes to the FDIAG RUNNING state. For modular services cards (MSCs), the service processor (SP) and CPU0 instance should go to the FDIAG RUNNING state. For distributed route processors (DRPs), the SP, CPU0 and CPU1 instances should all go to the FDIAG RUNNING state. Step 5 diagnostic start location node-id test {id |

all | basic | non-disruptive}

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/3/CPU0 test all

Starts the selected diagnostic tests.

Step 6 show diagnostic result location node-id [test {id | all}] [detail]

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/3/CPU0 test 1

Displays result of tests run at specified location. Possible results are shown as:

• Passed (.) • Failed (F) • Untested (U) Step 7 diagnostic unload location node-id

Example:

RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/3/CPU0

Unloads the offline diagnostic image, placing the specified card back in service.

Example:

(15)

Displaying Diagnostic Test Results

Displaying Diagnostic Test Results

To display test results for online diagnostics, perform the following procedure.

SUMMARY STEPS

1. admin

2. show diagnostic result location node-id [test {id | all}] [detail]

DETAILED STEPS

Displaying the Diagnostic Schedule

To display the diagnostic schedule, perform the following procedure.

SUMMARY STEPS

1. admin

2. show diagnostic schedule location node-id

Example:

Step 2 show diagnostic result location node-id [test {id | all}] [detail]

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/RP1/CPU0 test 1

Displays result of tests run at specified location. Possible results are shown as:

(16)

Cisco IOS XR Diagnostics Examples for Running Cisco IOS XR Diagnostics

DETAILED STEPS

Examples for Running Cisco IOS XR Diagnostics

This section contains the following examples:

• Running Health-Monitoring Diagnostics: Example, page 16

• Running Cisco IOS XR On-Demand Online Diagnostics: Example, page 17

• Scheduling Cisco IOS XR Online Diagnostics: Example, page 18

• Running Cisco IOS XR Integrated Field Diagnostics: Example, page 18

• Displaying Diagnostics Test Results: Example, page 21

Running Health-Monitoring Diagnostics: Example

In the following example, the generation of a syslog message when any health-monitoring test fails is enabled:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor syslog

In the following example, health-monitoring diagnostic test 1 for 0/1/CPU0 is enabled:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1

In the following example, health-monitoring diagnostic test 1 for 0/1/CPU0 is set to an interval of every 15 days at 6:00:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor interval location 0/1/cpu0 test 1 15 6:0:0.0

In the following example, the health-monitoring diagnostic testing failure threshold is set to 25 failures for test 1 on 0/1/CPU0:

RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor threshold location 0/1/CPU0 test 1 failure count 25

Example:

Step 2 show diagnostic schedule location node-id

Example:

RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/3/CPU0

(17)

Examples for Running Cisco IOS XR Diagnostics

Running Cisco IOS XR On-Demand Online Diagnostics: Example

In the following example, on-demand diagnostics are run on route processor (RP) card 0/RP1/CPU0. The tests available for the card are checked, then one of the available tests is run.

RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0 RP 0/RP1/CPU0:

Diagnostics test suite attributes:

M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA

P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA

A/I - Monitoring is active / Monitoring is inactive

Test Interval ID Test Name Attributes (day hh:mm:ss.ms shold) ==== ================================== ============ ================= ===== 1) ControlEthernetPingTest ---> *B*N****I 001 00:00:00.000 1 2) SelfPingOverFabric ---> *B*N****I 001 00:00:00.000 1 3) FabricPingTest ---> *B*N****I 001 00:00:00.000 1 4) ControlEthernetInactiveLinkTest -> *B*N****I 001 00:00:00.000 1 RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test all RP/0/RP1/CPU0:Jul 25 16:22:49.692 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetPingTest{ID=1} ... RP/0/RP1/CPU0:Jul 25 16:22:49.838 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: ControlEthernetPingTest{ID=1} has completed successfully RP/0/RP1/CPU0:Jul 25 16:22:49.844 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running SelfPingOverFabric{ID=2} ... RP/0/RP1/CPU0:Jul 25 16:23:19.948 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: SelfPingOverFabric{ID=2} has completed successfully RP/0/RP1/CPU0:Jul 25 16:23:19.951 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running FabricPingTest{ID=3} ... RP/0/RP1/CPU0:Jul 25 16:23:20.643 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: FabricPingTest{ID=3} has completed successfully RP/0/RP1/CPU0:Jul 25 16:23:20.646 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetInactiveLinkTest{ID=4} ...

RP/0/RP1/CPU0:Jul 25 16:23:20.927 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0:

ControlEthernetInactiveLinkTest{ID=4} has completed successfully

In the following example, an on-demand diagnostics test is stopped on RP card 0/RP1/CPU0:

RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test all

RP/0/RP1/CPU0:Jul 25 16:28:08.801 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetPingTest{ID=1} ... RP/0/RP1/CPU0:Jul 25 16:28:08.957 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: ControlEthernetPingTest{ID=1} has completed successfully RP/0/RP1/CPU0:Jul 25 16:28:08.963 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running SelfPingOverFabric{ID=2} ... RP/0/RP0/CPU0:router(admin)# diagnostic stop location 0/RP1/CPU0

(18)

%DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: SelfPingOverFabric{ID=2} has completed successfully RP/0/RP1/CPU0:Jul 25 16:28:39.068 : online_diag_hfr_rp[359]:

%DIAG-DIAG-6-DIAG_STOPPED : RP 0/RP1/CPU0: Diagnostic is stopped.

Scheduling Cisco IOS XR Online Diagnostics: Example

In the following example, no diagnostic test schedule is configured for 0/0/CPU0:

RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/0/CPU0 Current Time = Wed Nov 30 15:50:56 2005

Diagnostic for LC 0/0/CPU0 is not scheduled.

In the following example, diagnostic testing is scheduled for 0/0/CPU0:

RP/0/RP0/CPU0:router(admin)# configure

RP/0/RP0/CPU0:router(admin-config)# diagnostic schedule location 0/0/CPU0 test 1 daily 14:40

RP/0/RP0/CPU0:router(admin-config)# end

Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]: yes RP/0/RP0/CPU0:Nov 30 15:55:10.211 : config[65738]: %MGBL-LIBTARCFG-6-ADMIN_COMMIT : Administration configuration committed by user 'administrator'. Use 'show configuration commit changes 2000000001' to view the changes.

RP/0/RP0/CPU0:Nov 30 15:55:11.549 : config[65738]: %MGBL-SYS-5-CONFIG_I : Configured from console by administrator

RP/0/RP0/CPU0:router(admin)#

In the following example, the diagnostic test schedule for 0/0/CPU0 shows that test 1 will run daily at 14:40:

RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/0/CPU0 Current Time = Wed Nov 30 16:03:38 2005

Diagnostic for LC 0/0/CPU0: Schedule #1:

To be run daily 14:40

Test ID(s) to be executed: 1 .

Running Cisco IOS XR Integrated Field Diagnostics: Example

In the following example, the state of the nodes in the system are checked before running the integrated field diagnostics. The integrated field diagnostics are loaded on the card, placing the card out of service. While the diagnostics are running, the show platform command for the card is used to check that the diagnostics are running and the card is in the FDIAG RUNNING state. The diagnostics are then unloaded, returning the card to service.

RP/0/RP0/CPU0:router(admin)# show platform

(19)

RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/RP1/CPU0

diagnostic load will bring requested slot out of service. [confirm(y/n)] y User has confirmed diagnostic load request

Preparing UUT for Diagnostics software.

Downloading IDS diagnostics image /pkg/gsr/ucode/hfr-diag-rp-fdiags Please wait for UUT image downloading ...

diagnostic load in progress.

RP/0/RP0/CPU0:Nov 30 16:23:56.845 : shelfmgr[298]: %PLATFORM-SHELFMGR-3-USER_RESET : Node 0/RP1/CPU0 is reset due to user reload request

RP/0/RP0/CPU0:Nov 30 16:23:56.904 : redcon[287]: %HA-REDCON-6-STANDBY_CONNECTION_DOWN : connection to standby card is DOWN

RP/0/RP0/CPU0:Nov 30 16:23:57.444 : syncfs2[309]: %MEDIA-SYNCFS2-7-LRD_NOTAVAILABLE : Standby has been rendered UNAVAILABLE - STOP SYNC

RP/0/RP0/CPU0:Nov 30 16:24:11.462 : redcon[287]: %HA-REDCON-4-STANDBY_NOT_READY : standby card is NOT ready

RP/0/RP0/CPU0:Nov 30 16:24:35.072 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: INFO: CRS-1 RP Field Diagnostics V5.1]

Node Type PLIM State Config State ---0/0/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/0/CPU0 MSC 4OC192-POS/DPT IOS XR RUN PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A FDIAG RUNNING PWR,NSHUT,MON 0/SM0/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0

HQRP(FD) 0/RP1/CPU0:

Diagnostics test suite attributes:

M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA

P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA

A/I - Monitoring is active / Monitoring is inactive

Test Interval ID Test Name Attributes (day hh:mm:ss.ms shold) ==== ================================== ============ ================= =====

1) CRSLinecard.QuickTests ---> *B*D****I 000 00:00:00.000 n/a

2) CRSL3SP.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

3) CRSL3SP.FuncTests ---> ***D****I 000 00:00:00.000 n/a

4) L3SPTests.FuncTests ---> ***D****I 000 00:00:00.000 n/a

5) PPCModule.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

6) PPCModule.FuncTests ---> ***D****I 000 00:00:00.000 n/a

7) LCMotherboard.FuncTests ---> ***D****I 000 00:00:00.000 n/a

8) SharqModule.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

9) SprayerModule.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

10) SprayerModule.FuncTests ---> ***D****I 000 00:00:00.000 n/a

11) Sponge0Module.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

12) Sponge0Module.FuncTests ---> ***D****I 000 00:00:00.000 n/a

13) Sponge1Module.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

(20)

15) IngressMetroModul.MemoryTest ----> ***D****I 000 00:00:00.000 n/a

16) IngressMetroModul.FuncTests ---> ***D****I 000 00:00:00.000 n/a

17) EgressMetroModule.MemoryTest ----> ***D****I 000 00:00:00.000 n/a

18) EgressMetroModule.FuncTests ---> ***D****I 000 00:00:00.000 n/a

19) L3BIST.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

20) L3FCRAMMarch.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

21) PacketTests.FuncTests ---> ***D****I 000 00:00:00.000 n/a

22) 10GEPLIM(SP).FuncTests ---> ***D****I 000 00:00:00.000 n/a

23) PLIM8x10GE.MemoryTest ---> ***D****I 000 00:00:00.000 n/a

24) PLIM8x10GE.FuncTests ---> ***D****I 000 00:00:00.000 n/a

25) L3PacketTests.FuncTests ---> ***D****I 000 00:00:00.000 n/a

RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test 2

RP/0/RP0/CPU0:Nov 30 16:30:21.978 : online_diag_hfr_rp[341]: %DIAG-6-TEST_RUNNING : IFD 0/RP1/CPU0: Running Quick Test List{ID=2} ...

RP/0/RP0/CPU0:Nov 30 16:30:22.103 : online_diag_hfr_rp[341]: %HFRDIAG-3-ERROR : [UUT 0/RP1/CPU0: ERROR: Ids System - Slots - 33 RP - Ingress Module - External Memory - Bank 0 - Memory Address Pins Test: after writing byte pattern 20 to addr=0x00a00000, content at addr=0x00800000 changed from 0x00000000 to 0x14141414]

RP/0/RP0/CPU0:Nov 30 16:30:39.747 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: RESULT: UUT RP Field Diagnostics status: FAIL]

RP/0/RP0/CPU0:Nov 30 16:30:39.747 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: RESULT: tests_run=72, tests_fail=1, num_tests=72]

RP/0/RP0/CPU0:Nov 30 16:30:39.899 : online_diag_hfr_rp[341]: %DIAG-3-TEST_FAIL : IFD 0/RP1/CPU0: Quick Test List{ID=2} has failed. Error code = 0x1

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/RP1/CPU0 test 2 detail Current bootup diagnostic level for HQRP(FD) 0/RP1/CPU0: minimal

Test results: (. = Pass, F = Fail, U = Untested)

___________________________________________________________________________ 2 ) Quick Test List ---> F

Error code ---> 1 (DIAG_FAILURE) Total run count ---> 1

Last test execution time ----> Wed Nov 30 16:30:21 2005 First test failure time ---> Wed Nov 30 16:30:21 2005 Last test failure time ---> Wed Nov 30 16:30:21 2005 Last test pass time ---> n/a

Total failure count ---> 1 Consecutive failure count ---> 1

___________________________________________________________________________ RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/RP1/CPU0

RP/0/RP0/CPU0:Nov 30 16:39:07.711 : online_diag_hfr_rp[341]: %DIAG-6-GOLDXR_GENERAL : card_remove-d(529,-1)

RP/0/RP0/CPU0:Nov 30 16:39:07.711 : shelfmgr[298]: %PLATFORM-SHELFMGR-3-USER_RESET : Node 0/RP1/CPU0 is reset due to user reload request

RP/0/RP0/CPU0:Nov 30 16:39:27.740 : shelfmgr[298]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : 0/RP1/CPU0: MBI tftp:/hfr-os-mbi-3.3.80/mbihfr-rp.vm validated

RP/0/RP0/CPU0:Nov 30 16:41:27.615 : syncfs2[309]: %MEDIA-SYNCFS2-7-LRD_AVAILABLE : Standby has become AVAILABLE - START SYNC

RP/0/RP0/CPU0:Nov 30 16:41:28.354 : redcon[287]: %HA-REDCON-6-STANDBY_CONNECTION_UP : connection to standby card is UP

(21)

RP/0/RP0/CPU0:Nov 30 16:43:07.822 : redcon[287]: %HA-REDCON-4-STANDBY_READY : standby card is ready

RP/0/RP1/CPU0:Nov 30 16:43:04.877 : redcon[288]: %HA-REDCON-6-STBY_STANDBY_READY : This card is standby and is ready

Node Type PLIM State Config State ---0/0/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/0/CPU0 MSC 4OC192-POS/DPT IOS XR RUN PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A IOS XR RUN PWR,NSHUT,MON 0/SM0/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON

Displaying Diagnostics Test Results: Example

In the following example, the overall online diagnostics test results for 0/0/CPU0 are displayed:

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/0/cpu0 Current bootup diagnostic level for LC 0/0/CPU0: minimal

LC 0/0/CPU0:

Overall diagnostic result: PASS

Current bootup diagnostic level for LC 0/0/CPU0: minimal Test results: (. = Pass, F = Fail, U = Untested)

1 ) Control Ethernet Ping Test ---> .

In the following example, the detailed results of test 1 on 0/0/CPU0 are displayed:

RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/0/CPU0 test 1 detail

LC 0/0/CPU0:

Overall diagnostic result: UNTESTED Diagnostic level at card bootup: minimal Test results: (. = Pass, F = Fail, U = Untested)

___________________________________________________________________________

1 ) Control Ethernet Ping Test ---> .

Error code ---> 0 (DIAG_SUCCESS) Total run count ---> 1

Last test execution time ----> Wed Nov 30 17:05:21 2005 First test failure time ---> n/a

Last test failure time ---> n/a

Last test pass time ---> Wed Nov 30 17:05:21 2005 Total failure count ---> 0

Consecutive failure count ---> 0

(22)

Cisco IOS XR Diagnostics Additional References

Additional References

The following sections provide references related to Cisco IOS XR diagnostics.

Standards

MIBs

RFCs

Related Topic Document Title

Diagnostics commands: complete command syntax, command modes, command history, defaults, usage guidelines, and examples

Cisco IOS XR Interface and Hardware Components Command Reference, Release 3.4

Initial system bootup and configuration information for a router using Cisco IOS XR software

Cisco IOS XR Getting Started, Release 3.4

Standards Title

No new or modified standards are supported by this feature, and support for existing standards has not been modified by this feature.

—

MIBs MIBs Link

— To locate and download MIBs using Cisco IOS XR software, use the

Cisco MIB Locator found at the following URL and choose a platform under the Cisco Access Products menu:

http://cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml

RFCs Title

(23)

Additional References

Technical Assistance

Description Link

The Cisco Technical Support website contains thousands of pages of searchable technical content, including links to products, technologies, solutions, technical tips, and tools. Registered Cisco.com users can log in from this page to access even more content.

(24)