10.0.0.5 ping statistics
---3 packets transmitted, 0 packets received, 100% packet loss user@R6> show route 10.0.0.5
inet.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both
10.0.0.5/32 *[IS-IS/165] 00:02:39, metric 10 > to 10.1.26.1 via so-0/0/2.0
user@R6> traceroute 10.0.0.5
traceroute to 10.0.0.5 (10.0.0.5), 30 hops max, 40 byte packets 1 10.1.26.1 (10.1.26.1) 0.649 ms 0.521 ms 0.490 ms
2 10.1.26.2 (10.1.26.2) 0.521 ms 0.537 ms 0.507 ms 3 10.1.26.1 (10.1.26.1) 0.523 ms 0.536 ms 0.514 ms 4 10.1.26.2 (10.1.26.2) 0.528 ms 0.551 ms 0.523 ms 5 10.1.26.1 (10.1.26.1) 0.531 ms 0.550 ms 0.524 ms
Meaning
The sample output shows an unsuccessful ping command in which the packets are being rejected because the time to live is exceeded. The output for the show route command
Chapter 4: Troubleshooting
shows the interface (10.1.26.1) that you can examine further for possible problems. The traceroutecommand shows the loop between 10.1.26.1 (R2) and 10.1.26.2 (R6), as indicated by the continuous repetition of the two interface addresses.
Isolating the Causes of a Network Problem
Problem Description: A particular symptom can be the result of one or more causes. Narrow down the focus of your search to find each individual cause of the unwanted behavior.
Solution To isolate the cause of a particular problem, enter one or all of the following Junos OS CLI operational mode command:
To isolate the cause of a particular problem, enter one or all of the following Junos OS CLI operational mode command:
user@host> show < configuration | bgp | interfaces | isis | ospf | route >
Your particular problem may require the use of more than just the commands listed above. See the appropriate command reference for a more exhaustive list of commonly used operational mode commands.
Sample Output
user@R6>show interfaces terse
Interface Admin Link Proto Local Remote so-0/0/0 up up
so-0/0/0.0 up up inet 10.1.56.2/30 iso
so-0/0/2 up up
so-0/0/2.0 up up inet 10.1.26.2/30 iso
so-0/0/3 up up
so-0/0/3.0 up up inet 10.1.36.2/30 iso
[...Output truncated...]
The following sample output is from R2:
user@R2> show route 10.0.0.5
inet.0: 22 destinations, 25 routes (22 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both
10.0.0.5/32 *[Static/5] 00:16:21 > to 10.1.26.2 via so-0/0/2.0
[BGP/170] 3d 20:23:35, MED 5, localpref 100 AS path: 65001 I
> to 10.1.12.1 via so-0/0/0.0
Meaning
The sample output shows that all interfaces on R6 are up. The output from R2 shows that a static route [Static/5] configured on R2 points to R6 (10.1.26.2) and is the preferred route to R5 because of its low preference value. However, the route is looping from R2 to R6, as indicated by the missing reference to R5 (10.1.15.2).
Taking Appropriate Action for Resolving the Network Problem
Problem Description: The appropriate action depends on the type of problem you have isolated.
In this example, a static route configured on R2 is deleted from the [routing-options]
hierarchy level. Other appropriate actions might include the following:
Solution • Check the local router’s configuration and edit it if appropriate.
• Troubleshoot the intermediate router.
• Check the remote host configuration and edit it if appropriate.
• Troubleshoot routing protocols.
• Identify additional possible causes.
To resolve the problem in this example, enter the following Junos OS CLI commands:
[edit]
user@R2# delete routing-options static route destination-prefix user@R2# commit and-quit
user@R2# show route destination-prefix Sample Output
[edit]
user@R2# delete routing-options static route 10.0.0.5/32 [edit]
user@R2# commit and-quit commit complete
Exiting configuration mode user@R2> show route 10.0.0.5
inet.0: 22 destinations, 24 routes (22 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both
10.0.0.5/32 *[BGP/170] 3d 20:26:17, MED 5, localpref 100 AS path: 65001 I
> to 10.1.12.1 via so-0/0/0.0
Meaning
The sample output shows the static route deleted from the [routing-options] hierarchy and the new configuration committed. The output for the show route command now shows the BGP route as the preferred route, as indicated by the asterisk (*).
Evaluating the Solution to Check Whether the Network Problem Is Resolved
Problem Description: If the problem is solved, you are finished. If the problem remains or a new problem is identified, start the process over again.
You can address possible causes in any order. In relation to the network in“Isolating a Broken Network Connection” on page 42, we chose to work from the local router toward the remote router, but you might start at a different point, particularly if you have reason Chapter 4: Troubleshooting
to believe that the problem is related to a known issue, such as a recent change in configuration.
Solution To evaluate the solution, enter the following Junos OS CLI commands:
user@host> show route (ip-address |host-name) user@host> ping (ip-address | host-name) user@host> traceroute (ip-address | host-name) Sample Output
user@R6> show route 10.0.0.5
inet.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both
10.0.0.5/32 *[BGP/170] 00:01:35, MED 5, localpref 100, from 10.0.0.2 AS path: 65001 I
> to 10.1.26.1 via so-0/0/2.0 user@R6> ping 10.0.0.5
PING 10.0.0.5 (10.0.0.5): 56 data bytes
64 bytes from 10.0.0.5: icmp_seq=0 ttl=253 time=0.866 ms 64 bytes from 10.0.0.5: icmp_seq=1 ttl=253 time=0.837 ms 64 bytes from 10.0.0.5: icmp_seq=2 ttl=253 time=0.796 ms
^C
10.0.0.5 ping statistics
---3 packets transmitted, ---3 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.796/0.833/0.866/0.029 ms user@R6> traceroute 10.0.0.5
traceroute to 10.0.0.5 (10.0.0.5), 30 hops max, 40 byte packets 1 10.1.26.1 (10.1.26.1) 0.629 ms 0.538 ms 0.497 ms
2 10.1.12.1 (10.1.12.1) 0.534 ms 0.538 ms 0.510 ms 3 10.0.0.5 (10.0.0.5) 0.776 ms 0.705 ms 0.672 ms
Meaning
The sample output shows that there is now a connection between R6 and R5. The show routecommand shows that the BGP route to R5 is preferred, as indicated by the asterisk (*). The ping command is successful and the traceroute command shows that the path from R6 to R5 is through R2 (10.1.26.1), and then through R1 (10.1.12.1).
Routing Protocol Process Memory FAQs
The following sections present the most frequently asked questions and answers related to the routing protocol process memory utilization, operation, interpretation of related command outputs, and troubleshooting the software process.
Routing Protocol Process Overview
Junos OS is based on the FreeBSD Unix operating system. The open source software is modified and hardened to operate in the device’s specialized environment. For example, some executables have been deleted, while other utilities were de-emphasized.
Additionally, certain software processes were added to enhance the routing functionality.
The result of this transformation is the kernel, the heart of the Junos OS software.
The kernel is responsible for operating multiple processes that perform the actual functions of the device. Each process operates in its own protected memory space, while the communication among all the processes is still controlled by the kernel. This separation provides isolation between the processes, and resiliency in the event of a process failure. This is important in a core routing platform because a single process failure does not cause the entire device to cease functioning.
Some of the common software processes include the routing protocol process (rpd) that controls the device’s protocols, the device control process (dcd) that controls the device’s interfaces, the management process (mgd) that controls user access to the device, the chassis process (chassisd) that controls the device’s properties itself, and the Packet Forwarding Engine process (pfed) that controls the communication between the device’s Packet Forwarding Engine and the Routing Engine. The kernel also generates specialized processes as needed for additional functionality, such as SNMP, the Virtual Router Redundancy Protocol (VRRP), and Class of Service (CoS).
The routing protocol process is a software process within the Routing Engine software, which controls the routing protocols that run on the device. Its functionality includes all protocol messages, routing table updates, and implementation of routing policies.
The routing protocol process starts all configured routing protocols and handles all routing messages. It maintains one or more routing tables, which consolidate the routing information learned from all routing protocols. From this routing information, the routing protocol process determines the active routes to network destinations and installs these routes into the Routing Engine’s forwarding table. Finally, it implements routing policy, which allows you to control the routing information that is transferred between the routing protocols and the routing table. Using routing policy, you can filter and limit the transfer of information as well as set properties associated with specific routes.
Frequently Asked Questions: Routing Protocol Process Memory
This section presents frequently asked questions and answers related to the memory usage of the routing protocol process.
Why does the routing protocol process use excessive memory?
The routing protocol process uses hundreds of megabytes of RAM in the Routing Engine to store information needed for the operation of routing and related protocols, such as BGP, OSPF, IS-IS, RSVP, LDP and MPLS. Such huge consumption of memory is common for the process, as the information it stores includes routes, next hops, interfaces, routing policies, labels, and label-switched paths (LSPs). Because access to the RAM memory is much faster than access to the hard disk, most of the routing protocol process information is stored in the RAM memory instead of using the hard disk space. This ensures that the performance of the routing protocol process is maximized.
How can I check the amount of memory the routing protocol process is using?
Chapter 4: Troubleshooting
You can check routing protocol process memory usage by entering the show system processesand the show task memory Junos OS command-line interface (CLI) operational mode commands.
The show system processes command displays information about software processes that are running on the device and that have controlling terminals. The show task memory command displays memory utilization for routing protocol tasks on the Routing Engine.
You can check the routing protocol process memory usage by using the show system processescommand with the extensive option. The show task memory command displays a report generated by the routing protocol process on its own memory usage. However, this report does not display all the memory used by the process. The value reported by the routing protocol process does not account for the memory used for the TEXT and STACKsegments, or the memory used by the process’s internal memory manager. Further, the Resident Set Size value includes shared library pages used by the routing protocol process.
For more information about checking the routing protocol process memory usage.
For more information, see the show system processes command and the show task memorycommand.
I just deleted a large number of routes from the routing protocol process. Why is it still using so much memory?
The show system processes extensive command displays a RES value measured in kilobytes. This value represents the amount of program memory resident in the physical memory. This is also known as RSS or Resident Set Size. The RES value includes shared library pages used by the process. Any amount of memory freed by the process might still be considered part of the RES value. Generally, the kernel delays the migrating of memory out of the Inact queue into the Cache or Free list unless there is a memory shortage. This can lead to large discrepancies between the values reported by the routing protocol process and the kernel, even after the routing protocol process has freed a large amount of memory.
Frequently Asked Questions: Interpreting Routing Protocol Process-Related Command Outputs
This section presents frequently asked questions and answers about the routing protocol process-related Junos OS command-line interface (CLI) command outputs that are used to display the memory usage of the routing protocol process.
How do I interpret memory numbers displayed in the show system processes extensive command output?
The show system processes extensive command displays exhaustive system process information about software processes that are running on the device and have controlling terminals. This command is equivalent to the UNIX top command. However, the UNIX topcommand shows real-time memory usage, with the memory values constantly changing, while the show system processes extensive command provides a snapshot of memory usage in a given moment.
To check overall CPU and memory usage, enter the show system processes extensive command. Refer toTable 4 on page 49for information about the show system processes extensivecommands output fields.
user@host> show system processes extensive
last pid: 544; load averages: 0.00, 0.00, 0.00 18:30:33 37 processes: 1 running, 36 sleeping
Mem: 25M Active, 3968K Inact, 19M Wired, 184K Cache, 8346K Buf, 202M Free Swap: 528M Total, 64K Used, 528M Free
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
Table 4 on page 49describes the output fields that represent the memory values for the show system processes extensivecommand. Output fields are listed in the approximate order in which they appear.
Table 4: show system processes extensive Output Fields
Field Description Field Name
Information about physical and virtual memory allocation.
Mem
Memory allocated and actively used by the program.
Active
Chapter 4: Troubleshooting
Table 4: show system processes extensive Output Fields (continued)
Field Description Field Name
Memory allocated but not recently used or memory freed by the programs. Inactive memory remains mapped in the address space of one or more processes and, therefore, counts toward the RSS value of those processes.
Inact
Memory that is not eligible to be swapped, usually used for in-kernel memory structures and/or memory physically locked by a process.
Wired
Memory that is not associated with any program and does not need to be swapped before being reused.
Cache
Size of memory buffer used to hold data recently called from the disk.
Buf
Memory that is not associated with any programs. Memory freed by a process can becomeInactive, Cache, orFree, depending on the method used by the process to free the memory.
Free
Information about swap memory.
• Total—Total memory available to be swapped to disk.
• Used—Memory swapped to disk.
• Free—Memory available for further swap.
Swap
The rest of the command output displays information about the memory usage of each process. The SIZE field indicates the size of the virtual address space, and the RES field indicates the amount of the program in physical memory, which is also known as RSS or Resident Set Size. For more information, see the show system processes command.
What is the difference between Active and Inact memory that is displayed by the show system processes extensivecommand?
When the system is under memory pressure, the pageout process reuses memory from the free, cache, inact and, if necessary, active pages. When the pageout process runs, it scans memory to see which pages are good candidates to be unmapped and freed up.
Thus, the distinction between Active and Inact memory is only used by the pageout process to determine which pool of pages to free first at the time of a memory shortage.
The pageout process first scans the Inact list, and checks whether the pages on this list have been accessed since the time they have been listed here. The pages that have been accessed are moved from the Inact list to the Active list. On the other hand, pages that have not been accessed become prime candidates to be freed by the pageout process.
If the pageout process cannot produce enough free pages from the Inact list, pages from the Active list get freed up.
Because the pageout process runs only when the system is under memory pressure, the pages on the Inact list remain untouched – even if they have not been accessed recently – when the amount of Free memory is adequate.
How do I interpret memory numbers displayed in the show task memory command output?
The show task memory command provides a comprehensive picture of the memory utilization for routing protocol tasks on the Routing Engine. The routing protocol process is the main task that uses Routing Engine memory.
To check routing process memory usage, enter the show task memory command. Refer toTable 5 on page 51for information about the show task memory command output fields.
user@host> show task memory
Memory Size (kB) %Available When Currently In Use: 29417 3% now
Maximum Ever Used: 33882 4% 00/02/11 22:07:03 Available: 756281 100% now
Table 5 on page 51describes the output fields for the show task memory command.
Output fields are listed in the approximate order in which they appear.
Table 5: show task memory Output Fields
Field Description Field Name
Memory currently in use. Dynamically allocated memory plus theDATA segment memory in kilobytes.
Memory Currently In Use
Maximum memory ever used.
Memory Maximum Ever Used
Memory currently available.
Memory Available
The show task memory command does not display all the memory used by the routing protocol process. This value does not account for the memory used for the TEXT and STACKsegments, or the memory used by the routing protocol process’s internal memory manager.
Why is the Currently In Use value less than the RES value?
The show task memory command displays a Currently In Use value measured in kilobytes.
This value represents the memory currently in use. It is the dynamically allocated memory plus the DATA segment memory. The show system processes extensive command displays a RES value measured in kilobytes. This value represents the amount of program memory resident in the physical memory. This is also known as RSS or Resident Set Size.
The Currently In Use value does not account for all of the memory that the routing protocol process uses. This value does not include the memory used for the TEXT and the STACK segments, and a small percentage of memory used by the routing protocol process’s internal memory manager. Further, the RES value includes shared library pages used by the routing protocol process.
Any amount of memory freed by the routing protocol process might still be considered part of the RES value. Generally, the kernel delays the migrating of memory out of the Inactqueue into the Cache or Free list unless there is a memory shortage. This can lead to large discrepancies between the Currently In Use value and the RES value.
Chapter 4: Troubleshooting
Frequently Asked Questions: Routing Protocol Process Memory Swapping
This section presents frequently asked questions and answers related to the memory swapping of the routing protocol process from the Routing Engine memory to the hard disk memory.
How do I monitor swap activity?
When the system is under memory pressure, the pageout process reuses memory from the free, cache, inact and, if necessary, active pages. You can monitor the swap activity by viewing the syslog message reported by the kernel during periods of high pageout activity.
The syslog message appears as follows:
Mar 3 20:08:02 olympic /kernel: High pageout rate!! 277 pages/sec.
You can use the vmstat -s command to print the statistics for the swapout activity. The displayed statistics appear as follows:
0 swap pager pageouts 0 swap pager pages paged out
The swap pager pageouts is the number of pageout operations to the swap device, and the swap pager pages paged out is the number of pages paged out to the swap device.
Why does the system start swapping when I try to dump core using the request system core-dumpscommand?
The request system core-dumps command displays a list of system core files created when the device has failed. This command can be useful for diagnostic purposes. Each list item includes the file permissions, number of links, owner, group, size, modification date, path, and filename. You can use the core-filename option and the core-file-info, brief, and detail options to display more information about the specified core-dump files.
You can use the request system core-dumps command to perform a non-fatal core-dump without aborting the routing protocol process. To do this, the routing protocol process
You can use the request system core-dumps command to perform a non-fatal core-dump without aborting the routing protocol process. To do this, the routing protocol process