There are many applications that make network aware decisions. For instance the Resilient Overlay Network (RON) [ABKM01] monitors the delay and bandwidth on its networks and is able to provide delay or loss op- timized paths as required. Multimedia applications like [NN98] also rely on the delay information in order to construct their multi-cast routing tree. The knowledge of the delay is also useful in web caching [WY00] where the latency is used a metric to decide where a some content should be cached. The objective of this system is to cache data at geographically proximate locations, such that requests can be responded to with minimum delay. Peer-to-Peer like Bit-torrent [Bit] can also make intelligent choices about peers as studied in [Qur04]. Despite there being many potential AB aware applications, it is not clear what the performance gains would be on using an AB aware application.
There has also been some focus on designing applications which use the knowledge of AB. For instance SOBAS[DPJ04] relies on using the knowledge of the AB in order to adaptively set its socket buffer size, so as to optimize the performance of a transport protocol. This scheme works by probing for the AB and setting the socket buffer to the corresponding size. This scheme is particularly useful in high bandwidth networks, where congestion windows can become very large and exceeding the congestion window could cause a large number of packet drops. The scheme proposed helps maintain the congestion window to a level where there are very few packet drops. Dovrolis et al [ZDA06] also studied the performance of a scheme which could do bandwidth adaptive routing. The authors study the performance difference between doing proactive and reactive routing using the knowledge of the AB. However the authors do not address how they would obtain complete bandwidth information to implement this scheme. [TUAK04] utilizes the information about the AB
and the capacity to determine what the output rate of the streaming protocol should be. As can be observed there are several potential applications that could benefit from the knowledge of the AB. However there is no clear quantification as to what performance improvements could be obtained by using an AB aware application. In
this thesis we design an infrastructure that can monitor the AB of a network, which would give the above listed applications access to the AB information aiding them in making better network aware decisions.
CHAPTER 3
Evaluation of ABET implementations
We begin our series of evaluations by first evaluating publicly-available implementations of ABETs—in this first study, we treat each of these tools as a black-box and evaluate the default design choices along the algorithmic, implementation-related, and sampling-related temporal dimensions. A major focus of our analysis in this chapter is to understand the extent to which systemic issues and implementation efficiencies impact tool performance— the impact of temporal and algorithmic factors is studied in subsequent chapters.
ABETs face increasingly difficult measurement challenges as link speeds increase. Consider the issue of time precision: on faster links, time-gaps between packets decrease, rendering packet probe measurements more sensitive to timing errors. Available bandwidth measurements on high-speed links stress the limits of clock precision especially since additional timing errors may arise due to the NIC itself, the operating system, or the Network Time Protocol (designed to synchronize clocks of computers over a network) [PV02b]. Additionally, mechanisms such as interrupt coalescence that are used to improve network packet processing efficiency, mislead end-to-end tools that assume uniform per-packet processing and timing [PJD04].
On the other hand, ABETs are being increasingly deployed in high-speed network settings such as the Net- work Weather Service [Wol98] and the TeraGrid [Ter] infrastructure. Since the systemic issues mentioned above play a greater role in high-speed networks, it is critical to develop an understanding of the performance of promi- nent ABETs in such environments. Unfortunately, as described in Chapter 2, most past evaluations of ABETs have been restricted to paths with capacities of 100 Mbps or less—furthermore, these evaluations are often not comprehensive in the set of tools evaluated.
In this chapter we describe a comprehensive evaluation of ABETs in a high-speed network testbeds—to the best of our knowledge, this is the first such evaluation presented in the literature. We show that a comparative performance evaluation of various ABET implementations is inadvertently biased against certain tools since their performance is adversely impacted by interrupt coalescence and buffer size limitations (systemic biases) along with the inability of certain tool designs to detect these effects (implementation biases). We also run these tools on Internet paths and observe that the tools display similar performance characteristics to what we observed
in the test-bed setting. This implies that the results we obtain here are generalizable to the performance of the ABETs in actual deployments. It also presents strong motivations for designing and testing ABETs in high-speed testbeds similar to the one that we will describe in this chapter before deploying them on live systems.
3.1
Background: Interrupt Coalescence and network measurements
We first discuss how interrupt coalescence can impact network measurement and specifically AB estimation. Jain et. al. [PJD04] have conducted an in-depth analysis of this issue. Recall from Chapter 2 that AB estimation tools send a series of probe packets with a controlled and pre-determined spacing between them on the path of interest, and the receiver studies changes in the inter-packet gaps in order to make an inference about the AB . Clearly, ensuring high precision time-stamping and accurate spacing between the packets is critical for obtaining an accurate estimate of the AB.
To put this in context, in order to achieve a data rate of 1 Gbps, a 1500 byte packet would have to be transmitted (and received) every 12 µs. On general-purpose machines, packet transmissions and receipt by a network interface card (NIC) is handled by means of triggering interrupts—for instance, when a packet is received, the current process running on the CPU is interrupted, context is saved, and the packet is processed before returning the context to the original process. This processing and the associated context switch can be fairly costly operations to perform. Modern systems, consequently, reduce the overhead by grouping together packets that arrive close in time, and use a single interrupt to trigger their processing. This process is known as Interrupt coalescence (IC).
Unfortunately, since IC buffers packets arriving close together and uses a single interrupt to process all the packets, any timing information between the packets is lost—all such packets appear to have arrived back-to- back at the system. This can seriously limit the accuracy of AB estimation tools. There are no universal agreed upon parameters to setup IC. The network card that we used in our experiments had three parameters, which could be set. First, there is the maximum rate at which interrupts can be generated. Second, we can set the minimum time between a packet arriving and an interrupt being generated, which controls the maximum time a packet is queued at a device before it is acknowledged by the system. Finally, we can set the time between a packet arriving and a new interrupt being generated, which controls the minimum time a packet will spend in the queue. Using these parameters to control the IC can cause one of two effects in a pair of packets. First, if both the packets arrive between the interrupt intervals the two packets will be queued and acknowledged back-to-back
and will appear to have arrived with no spacing between them—this could cause compression in the spacing between the packets. The second situation is if the pairs of packets arrive with some spacing between them and the first packet is acknowledged by an interrupt before the second packet arrives. This would cause the second packet to stay in the queue for at least the minimum interrupt generation delay, which will increase the delay between the two packets. Thus, IC can distort the spacing between packets by increasing or decreasing it.
In [PJD04], the authors have proposed a scheme which can be used to detect IC and conduct AB estimation even in the presence of IC. This technique relies on the fact that a train of packet impacted by IC will show a saw-tooth like arrival process (as illustrated in Figure 2.3). The authors propose a heuristic to identify the spikes and use only the first observation from each spike to make the estimates of the AB.
The above discussion highlights the fact that IC can have a significant impact on the process of AB estimation. A part of our evaluation in this chapter, we illustrate the impact that IC can have on some AB estimation tools and discuss why certain techniques are more sensitive to the effects of IC on contemporary computers.