MBNI Network Performance: Second Report
Ali Asad Lotia
1
Introduction
This report is an analysis of the results obtained from network performance tests conducted on 12/19/2006. The testing was done between machines connected at several points along the network to facilitate the detection of any perfor-mance bottlenecks. The simplified network paths as well as the highest average throughput results for iperf TCP and CIFS are given on the included diagram. Additionally I have also provided graphical representations of the test results to aid in the evaluation of the performance.
2
Test details
The test machines were connected as shown in the network map below (see appendix for larger sized image). The file transfer rates between the remote collaboration sites and the MBNI file server need to be improved and so it was used as the main measurement node in these tests. The client on the local private subnet behind the Checkpoint firewall was a laptop running Windows XP (laptop) which was assigned a static private IP address. The Windows 2003 server (server) and this laptop were on the same subnet and so there were no routers, firewalls or NAT devices between these two machines. Further testing was done with three machines outside the MBNI firewall: a Mac mini (cakebox) connected to the same switch as the firewall; a Dell desktop PC (et-linux) running Linux located at 2055 Palmer Commons which was connected to an MCIT switch 2 router hops from the Windows file server ; and a server running Linux (ntap1) located at the Merit Gigapop. I also tested paths that did not include the server to obtain data for comparison.
I used Samba on the Linux and Mac OS X machines and mounted the share over IP on the server to allow CIFS file transfers between nodes and emulate the data transfers between the remote collaboration sites. Since I didn’t have CIFS a benchmarking tool, I used a stopwatch to time large file transfers between machines to obtain approximate throughput values. I also used iperf in UDP and TCP modes to measure raw network throughput to establish “best case” performance.
3
Observations
3.1
iperf Results
A plot of the results is given in the figures below (larger sized versions of this figure are included as appendices).
Strangely, UDP throughput is substantially lower than TCP on all segments which may be due to network misconfiguration, a bug in iperf or possibly due to both. Because the UDP results are so low, I have restricted my observations to the TCP results until I know the reason for the poor UDP performance.
The TCP throughput between the laptop and server behind the MBNI fire-wall is over 700 Mbps which meets the expected throughput for machines on the same subnet. It is important to note that for TCP file transfers, window size makes a very large difference in performance. We were able to get a greater than 2x improvement by increasing the TCP window from 8KB to 128KB on both client and server machines. Throughput values dropped substantially to approx-imately 150 Mbps when testing against machines across the firewall. Testing against the cakebox which was connected on a different VLAN on the same switch as the server provided near identical results as testing against the et-linux machine which is two router hops away from the server. Throughput to the ntap1 machine at the Merit gigapop was only 10% ( 70 Mbps) of the local subnet throughput.
To eliminate the possibility that the traffic could be throttled by routers and switches in the path, I also tested the paths in which neither machine was behind the MBNI firewall. The throughput between the cakebox and the et-ubuntu machine averaged over 600 Mbps. The cakebox also averaged approximately 400 Mbps when sending to the ntap1 machine at the Merit Gigapop which shows that neither one of these two network paths are limiting the throughput.
3.2
CIFS results
The bar chart below shows the approximate throughput during CIFS file trans-fers (a larger version is included as an appendix). As expected, the CIFS values are lower than the iperf throughput since iperf generally provides “best case” results which do not account for the overheads present in most network appli-cations. Nevertheless, the discrepancies between the CIFS and iperf results are surprisingly large. The CIFS throughput values are less than 30% of the iperf values between the client and server machines on the MBNI private subnet. Firewall traversal in CIFS file transfers does not result in as great a perfor-mance hit as was encountered with iperf but still causes a noticeable drop in speed as shown in the second bar in the chart. Additional routers in the net-work path also decrease CIFS file transfer performance which is consistent with expectations.
4
Conclusions and Recommendations
It is clear that the MBNI firewall has a significant impact on network perfor-mance. As noted above, iperf throughput suffered more than CIFS throughput when traversing the firewall. The firewall configuration should be tuned to minimize the impact it has on authorized traffic.
Elvis Jakupovic mentioned that MBNI did not currently have plans to move to 10 Gbps network interfaces on their servers so the best possible performance that can be expected is in the 900 Mbps range. Taking into account the currently obtained CIFS transfer rates, the current links provide considerable headroom for improvement in performance. Tuning the Windows file server to optimize CIFS file transfer speeds may also be possible although I have not looked into this. The group may want to consider a switch to Linux since NFSv4 perfor-mance out of the box is considerably better than the observed CIFS perforperfor-mance and the on-the-wire-data can be encrypted using several different ciphers.
5
Appendices
The network diagram and plots of observed network performance are provided on the following pages.
Stanford
Firewall
MBNI
Palmer Commons
Linux@PC
Mac mini
Cakebox
Win2003
Server
T
est WinXP
laptop
?/?
MCIT
Merit
Gigapop
ntap1.merit.edu
Created on Mon Dec 18 2006
Modifi
ed on
Thu Jan 1
1 2007
MBNI Network Performance
Key
Network path
Max iperf
Throughput/Max
CIFS
Throughput
(Mbps)
722/203.95
598/?
391/?
329/?
150/136
150/102.489
