On a typical server, network performance is as important as disk, memory, and CPU performance. After all, the data has to be delivered over the network to the end user. The problem, however, is that things aren’t always as they seem. In some cases a net- work problem can be caused by misconfiguration in server RAM. If, for example, packets get dropped on the network, the reason may very well be that your server doesn’t have enough buffers reserved for receiving packets, which may be because your server is low on memory. Again, everything is related, and your task is to find the real cause of the troubles.
When considering network performance, there are different kinds of information to be analyzed. As you know, several layers of communication are used on the network. If you want to analyze a problem with your Samba server, that requires a completely different approach from analyzing a problem with dropped packets. A good network performance analysis always goes from the bottom up. That means that you first need to check what is happening at the physical layer, and then go up through the Ethernet, IP, TCP/UDP, and protocol layers.
When analyzing network performance, always start by checking the network inter- face itself. Good old eb_kjbec offers excellent statistics to do just that. For instance, consider Listing 3-19, which gives the result of eb_kjbec on the apd- network interface.
Listing 3-19. Use ifconfig to See What Is Happening on Your Network Board
nkkp<iah6zeb_kjbecapd- apd-Hejgaj_]l6ApdanjapDS]``n,,6,_6b26/b61`6^^ ejap]``n6-,*,*,*-,>_]op6-,*,*,*.11I]og6.11*.11*.11*, ejap2]``n6ba4,66.,_6b2bb6ba/b61`^^+20O_kla6Hejg QL>NK=@?=OPNQJJEJCIQHPE?=OPIPQ6-1,,Iapne_6- NTl]_gapo634-1/5,annkno6,`nklla`6,krannqjo6,bn]ia6, PTl]_gapo6-.0.34.2annkno6,`nklla`6,krannqjo6,_]nnean6, _khheoekjo6,ptmqaqahaj6-,,, NT^upao6144,14540$12,*4I>%PT^upao6-1.2503-3-$-*0C>% Ejpannqlp6-4>]oa]``naoo6,ta4,,
As you can see from Listing 3-19, the apd- network board has been quite busy, with 560 MB of received data and 1.4 GB of transmitted data. This is the total overview of what your server has been doing since it started up, so you will see that these statistics can be much higher for a server that has been up and running for a long time. You can also see that IPv6 (ejap2) has been enabled for this network card. There’s nothing wrong with that, but if you don’t use IPv6, there’s no reason why it should be enabled.
Next, in the lines NTl]_gapo and PTl]_gapo, you can see send (transmit, TX) and receive (RX) statistics. It’s not especially the number of packets that is of interest here, but mainly the number of erroneous packets. In fact, all of these parameters should be 0 at all times. If you see anything else, you should check what is going on. The following error indicators are displayed using eb_kjbec:
sannkno: Represents the number of packets that had an error. Typically, this is due to bad cabling or a duplex mismatch. In modern networks, duplex settings are detected automatically, and most of the time that goes quite well, so if you see an increasing number here, it might be a good idea to replace the patch cable to your server.
s`nklla`: A packet gets dropped if the server has no memory available to receive it. Dropped packets will also occur on a server that runs out of memory, so make sure that you have enough physical memory installed in your server.
skrannqjo: An overrun will occur if your NIC gets overwhelmed with packets. If you are using up-to- date hardware, overruns may indicate that someone is doing a denial-of-service attack on your server.
sbn]ia: A frame error is an error caused by a physical problem in the packet, such as a CRC error. You may see this error on a server with a bad connection link.
s_]nnean: The carrier is the electrical wave that is used for modulation of the signal. It really is the component that carries the data over your network. The error coun- ter should be 0 at all times, and if it isn’t, you probably have a physical problem with the network board, so it’s time to replace the network board itself.
s_khheoekjo: You might see this error in an Ethernet network in which a hub is used instead of a switch. Modern switches make packet collisions impossible, so you will probably never see this error anymore.
Listing 3-20. Use ethtool to Check Settings of Your Network Board nkkp<iah6zapdpkkhapd- Oappejcobknapd-6 Oqllknpa`lknpo6WPLY Oqllknpa`hejgik`ao6-,^]oaP+D]hb-,^]oaP+Bqhh -,,^]oaP+D]hb-,,^]oaP+Bqhh -,,,^]oaP+Bqhh Oqllknpo]qpk)jackpe]pekj6Uao =`ranpeoa`hejgik`ao6-,^]oaP+D]hb-,^]oaP+Bqhh -,,^]oaP+D]hb-,,^]oaP+Bqhh -,,,^]oaP+Bqhh =`ranpeoa`]qpk)jackpe]pekj6Uao Olaa`6-,,,I^+o @qlhat6Bqhh Lknp6Pseopa`L]en LDU=@6, Pn]jo_aeran6ejpanj]h =qpk)jackpe]pekj6kj OqllknpoS]ga)kj6lqi^c S]ga)kj6c ?qnnajpiaoo]caharah6,t,,,,,,//$1-% Hejg`apa_pa`6uao
Typically, just a few parameters from the apdpkkh output are of interest, the Olaa` and
@qlhat settings. They show you how your network board is talking to other nodes. If you see, for example, that your server is set to full duplex, whereas all other nodes in your net- work use half duplex, you’ve found your problem and know what you need to fix.
Another nice tool to monitor what is happening on the network is IPTraf (start it by entering elpn]b). This is a real- time monitoring tool that shows what is happening on the network from a graphical interface. When you start it, it shows you a license agreement window. From that window, press a key to continue to the IPTraf main menu, which you can see in Figure 3-1.
Before you launch IPTraf from this menu, choose the Configure option. From there, you can specify what exactly you want to see and how you want it to be displayed. For instance, a useful setting to change is the additional port range. By default, IPTraf shows activity on privileged TCP/UDP ports only. If you have a specific application that you want to monitor and it doesn’t use one of these privileged ports, select Additional Ports from the configuration interface (see Figure 3-2) and specify additional ports that you want to monitor.
After telling elpn]b how to do its work, from the main menu use the IP Traffic Moni- tor option to start the tool. You can next select on which interface you want to listen, or just press Enter to listen on all interfaces. Next, IPTraf asks you in which file you want to write log information. You should be aware that it isn’t always a good choice to configure logging, because logging may fill up your file systems quite fast. In case you don’t want to log, press Ctrl+X now. This will start the IPTraf interface, which displays everything that is happening on your server and on what port exactly it is happening (see Figure 3-3).
Figure 3-3. IPTraf gives a real- time overview of what is happening on your server’s network boards.
Apart from the real- time overview of what is happening on the network, IPTraf also offers the LAN station monitor, shown in Figure 3-4. This interface is a great help in find- ing workstations that cause a lot of network load. For instance, you can use this to find the workstation that is doing video streaming or online gaming.
Figure 3-4. The LAN station monitor shows the most active LAN station at the top of the list.
If it’s not so much the performance on the network card that you are interested in, but more what is happening at the service level, japop]p is a good basic network perfor- mance tool. It uses different parameters to show you what ports are open and on what ports your server sees activity. My personal favorite way of using japop]p is by issuing the
japop]p)pqhlj command. This gives an overview of all listening ports on the server and even tells you what other node is connected to a particular port. See Listing 3-21 for an overview.
Listing 3-21. netstat Enables You to See Which Ports Are Listening on Your Server and Who Is Connected
nkkp<iah6zjapop]p)pqhlj
=_peraEjpanjap_kjja_pekjo$kjhuoanrano%
p_l,,,*,*,*,6---,*,*,*,6&HEOPAJ0304+lknpi]l p_l,,,*,*,*,64,,*,*,*,6&HEOPAJ114.+]l]_da. p_l,,,*,*,*,60.4/1,*,*,*,6&HEOPAJ1/54+nl_*ikqjp` p_l,,-.3*,*,*-610/.,*,*,*,6&HEOPAJ1-53+lkopcnao p_l,,-.3*,*,*-62,-,,*,*,*,6&HEOPAJ15/4+- p_l,,,*,*,*,6//.0.,*,*,*,6&HEOPAJ) p_l,,,*,*,*,62,,.2,*,*,*,6&HEOPAJ0320+nl_*op]p` p_l,,,*,*,*,600/,*,*,*,6&HEOPAJ1.5,+]l]_da. p_l,,,*,*,*,6001,*,*,*,6&HEOPAJ10-5+oi^` p_l,,,*,*,*,6225,*,*,*,6&HEOPAJ1-12+uloanr p_l2,,666..666&HEOPAJ1--3+ood` p_l2,,66-62,-,666&HEOPAJ15/4+- q`l,,,*,*,*,6.,05,*,*,*,6&) q`l,,-5.*-24*-*556-/3,*,*,*,6&10-3+ji^` q`l,,-,*,*,*-,6-/3,*,*,*,6&10-3+ji^` q`l,,,*,*,*,6-/3,*,*,*,6&10-3+ji^` q`l,,-5.*-24*-*556-/4,*,*,*,6&10-3+ji^` q`l,,-,*,*,*-,6-/4,*,*,*,6&10-3+ji^` q`l,,,*,*,*,6-/4,*,*,*,6&10-3+ji^` q`l,,,*,*,*,6224,*,*,*,6&1-12+uloanr q`l,,,*,*,*,623-,*,*,*,6&1-15+nl_*ull]oos`` q`l,,,*,*,*,6230,*,*,*,6&1-2.+nl_*ultbn` q`l,,,*,*,*,624.,*,*,*,6&1-3,+ul^ej` q`l,,,*,*,*,624/,*,*,*,6&1-3,+ul^ej` q`l,,,*,*,*,60131,,*,*,*,6&) q`l,,,*,*,*,63,,,*,*,*,6&0320+nl_*op]p` q`l,,,*,*,*,623,*,*,*,6&11.4+`d_l`/ q`l,,,*,*,*,625,*,*,*,6&10.4+ej*pbpl` q`l,,,*,*,*,6/1,-2,*,*,*,6&0320+nl_*op]p` q`l,,,*,*,*,601,-,,*,*,*,6&1/54+nl_*ikqjp` q`l,,,*,*,*,6---,*,*,*,6&0304+lknpi]l q`l,,-,*,*,*-,6-./,*,*,*,6&012/+jpl` q`l,,-5.*-24*-*556-./,*,*,*,6&012/+jpl` q`l,,-.3*,*,*-6-./,*,*,*,6&012/+jpl` q`l,,,*,*,*,6-./,*,*,*,6&012/+jpl` q`l2,,ba4,66.-56`-bb6baa`6-./666&012/+jpl` q`l2,,ba4,66.,_6b2bb6ba/b6-./666&012/+jpl` q`l2,,66-6-./666&012/+jpl` q`l2,,666-./666&012/+jpl`
When using japop]p, quite a few options are available. The following is an overview of the most interesting ones:
s)l: Shows the PID of the program that has opened a port
s)_: Updates the display every second
s)o: Shows statistics for IP, UDP, TCP, and ICMP
s)p: Shows TCP sockets
s)q: Shows UDP sockets
s)s: Shows RAW sockets
s)h: Shows listening ports
s)j: Resolves addresses to names
Many other tools are available to monitor the network as well, but most of them are beyond the scope of this chapter because they are protocol or service specific; thus, they won’t help you as much in finding performance problems on the network. However, I want to mention one very simple performance testing method that I personally use at all times when analyzing a performance problem. Because all that counts when analyz- ing network performance is how fast your network can copy data from and to your server, I like to measure that by creating a big file (1 GB, for example) and copying it over the network. To measure time, I use a peia command that gives a clear impression of how long it really took to copy the file. For instance, peiao_loanran6+^ecbeha+hk_]h`en will end with a summary of the total time it took to copy the file over. This is an excellent test, especially when you start optimizing performance, because it will show you immediately whether you reached your goals or not.