1
Project Proposal
Project title: Statistical Analysis of Internet Data. Project acronym: SAID.
Principal Investigator: Dr. G. Hooghiemstra.
2
Summary
Internet was designed about thirty years ago. In retrospect, its successful operation up to now is highly remarkable in view of itsbest-effortservice. As current communication is rapidly evolving towards multimedia, the architecture and network topology of Internet should be modified to offer the desired quality of service (QoS) for each type of medium.
Many questions about the performance of Internet triggered this project. We mention a few: 1. What kind of graphs are reasonable models for the Internet graph?
2. Growth of the Internet: Is the growth exponential in time, and how can this be measured? 3. How can we model end-to-end delays? Is the observed ”long range dependence” present through the entire Internet? Is there any connection between the delay and the hopcount? We intend to address these questions by relating empirical evidence obtained from Internet data to mathematical, in particular stochastic, models of the Internet. Obviously, a successful model will not only enhance our understanding of the current phenomena in Internet, but will lead to recommendations for improvements on both the network infrastructure and the network protocols.
Two ingredients are defining a network: the topology or interconnection between nodes (routers or switches), and the specification of the links in terms of QoS qualifiers. The project is composed of two parts. In Part I, we investigate the topological structure of the Internet. In Part II, we study end-to-end delays, which are connected to the QoS qualifiers mentioned above.
Our first research topic is to describe the graph of the Internet with a hierarchical model (see 6.4.1). We propose to proceed by testing this model on its multicast efficiency and on available data (see Section 6.3). The final and third topic in this topological part will investigate whether the Internet as a graph has an exponential structure or not.
The second part involves the modeling of delays as seen by a probe-packet on a single source-destination pair. This delay consists of processing delay (through the routers) and queueing delay of other Internet traffic. Using lab measurements we are able to identify the processing delay. We will try to model the queueing delay and will compare the model with the avail-able measurements from RIPE NCC. Further goals are to monitor the delay over time, and to determine the interaction of the delay.
3
Composition of the research group
The following table comprises the members of the research group, their e-mail addresses, spe-cialism, and the number of hours per week that they will spent on the project.
Dr. R. W. van der Hofstad Dr. G. Hooghiemstra
[email protected] [email protected] Prob. theory & Statistical Physics Probability theory & Statistics
8 8
Prof. dr. ir. P. Van Mieghem Oio’s and postdoc (vacancies) [email protected]
-El. Eng. and Comp. science El. Eng./Probability Theory
8 full time.
Address Hooghiemstra and Van Mieghem: Address Van der Hofstad: Delft University of Technology Technical University Eindhoven
Information Technology and Systems Faculty of Mathematics and Computer Science P.O. Box 5031, 2600 GA Delft, P.O. Box 513, 5600 MB Eindhoven
The Netherlands The Netherlands.
4
Research school
Thomas Stieltjes Institute of Mathematics.
5
Required funding period
Start of the project: September 2002.
Length: 4 years, 2 Ph.D.-students, and one postdoc for 212 years (see Section 8).
6
Description of the proposed research
6.1
Introduction
We start with a short introduction of the research group and their previous work. The research group is multi-disciplinary. Since for modeling good measurements are of the utmost importance, we have included an extended section on measurements. The actual proposal consists of two parts: I. Topological structure and II. End-to-end delays.
6.2
Previous work of the research group
We started to cooperate in the fall of 1999 with the study of graph properties of the Internet. This resulted in the papers [11], [14] and a conference proceedings [15]. In these papers, we have tried to find a suitable model that has similar properties as the Internet. The model was the random graph with exponential weights and shortest path communication between the nodes. This model has similar properties in terms of the hopcount, but has rather different nodal degrees. One can think of this model as being built up from two main ingredients (1) A birth process that describes at any time how many nodes can be reached with a sum of link weights less than the specified time, and (2) recursive trees, which describe how the points of the birth process should be joined together. As we are interested in the number of hops between any two
points, we are naturally lead to recursive trees. For more discussion on Internet models, see Section 6.4.1 below.
A continuation of this research is our paper on multicast, which will appear as [16]. In this paper, we have investigated the average gain by using the multicast procedure compared to unicast. We have obtained exact formulas for this gain both fork-regular trees as well as for the random graph model with exponential weights.
As a side result in studying almost sure behavior of random graphs, we obtained [12], which contains the covariance between the level sets of recursive trees.
The research group has expertise in queueing, extreme values (regular varying behavior, fat tails), limit theory and statistics (Hooghiemstra); in statistical physics (polymers and percolation theory) and large deviations (Van der Hofstad); and in telecom-networking and protocols (Van Mieghem). We believe it to be one of our strengths to have such diverse background and expertise. Moreover, the ideas in statistical physics appear to be quite relevant in the analysis of networks (see e.g., the Self-Organizing Networks Group at the Notre Dame Physics Department, which studies measurements on the Internet and the Web graph).
6.3
Required measurements
6.3.1 Relation to RIPE NCC and UMEEPI
RIPE NCC, the Network Coordination Centre of the R´eseaux IP Europ´een, is continuously measuring the delay and the hopcount of IP-packets transmitted between fixed measurement boxes in some part of the Internet. At this moment (Fall 2001), about 40 measurement boxes are scattered mainly over Europe. Between each pair of measurement boxes, small IP packets of a fixed length (100 bytes), called probe-packets, are transmitted with interarrival times of about 40 seconds, resulting in a total of about 2160 probe-packets per day. The sending measurement box generates an accurate time-stamp synchronized via GPS in each probe-packet, while the receiving measurement box reads the GPS-time of the probe-packet upon arrival. The end-to-end delay or one-way transit time of probe-packets is defined as the difference between these two time-stamps and has an accuracy of 10µs. At regular times (every 6 minutes), path information (the number of hops, IP addresses of intermediate routers) between each pair of measurement boxes is obtained from the trace-route utility. The specific details of the RIPE NCC measurement configuration are described in [10].
The data obtained by RIPE is fully at our disposal as long as the privacy of the different server providers is guaranteed; our contacts with RIPE NCC run through Dr. Henk Uijterwaal. Since the end of 2000, Van Mieghem and Hooghiemstra became members of a group of scien-tists who want to employ the RIPE NCC data to enlarge the knowledge of the Internet. The name of this group is UMEEPI (Understanding and Modeling End-to-End Properties in Inter-net). The group consists of scientists from the University of Amsterdam (Computer science); Delft University of Technology (both Management Science, Electrical Engineering and Probabil-ity/Statistics); KPN-Research and RIPE NCC. Through these meetings which take place every three months it is possible to influence the future measurements of RIPE. For example therouter delay measurements at RIPE NCC described below were performed shortly after the UMEEPI meeting of June 2001 as a result of a presentation of our delay analysis.
Finally, RIPE NCC is also planning new measurement designs. Through our cooperation we can influence the directions of these measurements.
6.3.2 Other measurements
Apart from the extensive data from RIPE NCC that we will have at our disposal, we will continue to gather data from other sources. Currently, we have a large data set corresponding to hopcounts, where also the number of hops in the autonomous system domains(AS) are recorded. This data will be useful in order to investigate the hierarchical model for the Internet based on difference between the autonomous domains and the Backbone.
We will not hesitate to ask for data sets that are described in relevant papers appearing in the literature. In the past, we have been able to use data sets from other sources, due to the fact that the authors kindly agreed that we could use them. For instance, the multicast measurements of [20] have been used in [16]. The raw data has been kindly put to our disposal by Mr. Tangmunarunkit and has been used to model multicast trees.
6.4
Project plans
In this section, we will formulate questions that we intend to address. This section is divided into two parts. the first part deals with the investigation of the topological structure of the Internet, whereas the second part deals with end-to-end delays.
6.4.1 Part I: Topological structure
We intend to further investigate the topological structure of the Internet. The Internet is a vast graph, and it is virtually impossible to calalogue the graph completely. Moreover, it is also difficult to obtain information about the Internet, due to the fact that many providers wish to keep the properties of their part of the Internet confidential. However, Internet data is available (see Section 6.3). One of the best ways to learn about the Internet is to perform trace-routes between many pairs of nodes on the Internet. Of course, this will lead to indirect data, from which it is difficult to deduce topological properties of the Internet.
We have data sets that give trace-routes between many pairs of vertices of the Internet. These data sets are of two kinds: (a) between one fixed node to many destinations, and (b) between many pairs of nodes. In Case (a), we see the Internet from a single source, whereas in Case (b), we see the Internet from multiple sources. We now come to ouruniformity assumption. In Case (a), we will assume that the destinations are uniform over the Internet, wheras in Case (b) we will assume that the pairs will be uniform pairs of nodes on the Internet. This assumption is necessary in order to perform a statistical analysis of the data. Moreover, in acquiring data sets, we will always try to satisfy the uniformity assumption as closely as possible.
As a basic assumption, we will use that the hopcount on the Internet is close to being a Poisson random variable. Indeed, many measurements (see e.g., [18],[21] and [2]) indicate that (1) the mean and variance of the hopcount distribution are almost identical, and (2) the hopcount distribution is close to being a normal distribution. Note that for a Poisson distribution, both properties hold. Moreover, the parameter of the Poisson distribution is equal to the mean hopcount. We will refer to graphs having the above property as having the “Poisson property”. There are many models which have the Poisson property. For instance, the random and the complete graph with exponential weights have the Poisson property (see [14]). Moreover, also many tree type models satisfy the assumption (see e.g., [6]). However, it appears that a graph needs to be somewhat irregular to satisfy the Poisson property. In fact, many random graphs satisfy the Poisson property, whereas we know of no deterministicgraph that satisfies it.
Taking the above considerations into account, we will try to model the Internet as a random graph satisfying the Poisson property. Below we will give further indications of what we intend to study precisely.
Autonomous System (AS) domains versus IP.
We have extensive measurements of the number of hops in the autonomous domains and the IP routers. It turns out that the number of IP routers per AS domain follows a power law.
We propose to use a hierarchical model that describes these results in a satisfying way. The two levels of the hierarchical model are the AS level and the IP level. We can assume that the Internet viewed from the level of AS domains is more strongly interconnected, in the sense that there are quite a number of links between the different autonomous domains. Therefore, a model based on a random graph with exponential weights may not be unreasonable. At every autonomous domain, we have located a number of IP routers, where the number follows a power law as observed from the measurements. Note that when the number of autonomous domains is small compared to the number of IP routers, that most of the local properties of the Internet are determined by the IP level in the hierarchical model. We will study the hopcount distribution at those two levels. Moreover, we will propose and investigate estimators of the relevant parameters in the model, such as the size of the Internet and the number of autonomous domains.
The Internet as a power law graph.
In [8], the degree sequence of the Internet was investigated. The degree sequence is the sequence that contains the number of sites with degreek for every integerk. In [8], it was shown that this degree sequence follows a power law. Indeed, it was shown that Dk ≈N k−γ for k large, with γ
between 2.16 and 2.20. The above is true as long as the k is much smaller than the diameter of the graph. It is at this point that our first model, the random graph with exponential weights on the links, is really different from the Internet. We can show that the expected number of points with degree k is approximately N P(Z =k), where Z is a normal random variable with mean logN and variance logN. Therefore, the degree sequence of our first model is far from a power law.
As described above, the degree sequence is largely determined by the model for the autonomous domains, since the number of autonomous domains is small compared to the number of IP routers. We propose to investigate models for the autonomous domains which have both the Poisson property, and have a power law degree sequence. A candidate for such a model is a tree type model (as in [6]). It is not unreasonable to expect that the autonomous domains are very little interconnected, and are therefore close to trees.
An important ingredient to the above work will be to devise a method to estimate the relevant parameters from available data. Indeed, given our model that will contain several parameters (such as the size of the graphN, the relative size of the number of AS domains and the power law exponents), we should find good estimators of these parameters from the data.
Multicast efficiency.
The above model will hopefully provide a reasonable model for the Internet. In fact, many of the properties of the Internet are built in. Here we think of the Poisson property, the hierarchy of the Internet coming from AS versus IP, and the power law degree sequence. A test to see whether the model is a good candidate to model the Internet is to study its multicast properties. In [16], we have investigated such properties for general graphs. Moreover, we have data concerning the number of simultaneous hops necessary in a multicast session with an arbitrary number of multicast members. We can then compare these with the predicted multicast properties in the
On the exponential growth of the Internet and the diameter of the Internet.
A question that appears quite frequently is whether the Internet is a graph that grows exponen-tially. Exponential growth of a graph can be translated into the question whether the number of points that can be reached grows exponentially in the number of steps. Another formulation is that the Internet contains a uniformly growing tree.
On one hand, since trees are very efficient in reducing the number of hops needed for a typical message, one would hope, maybe even expect, that the Internet is exponentially growing. On the other hand, exponential growth of a graph is a global property that is not so easily measured. We wish to study practical ways to measure whether real-life graphs are exponentially growing and test these procedures upon Internet data.
A problem with exponential growth of graphs is that these are often defined in terms of the exponential growth in k of the number of points that can be reached in k steps. For a finite graph, this does not really make sense. Instead, in [16], we proposed to take the logarithmic growth inN of the mean hopcount as a definition. Moreover, this allows us to define the growth constant askc =elogN/E(HN), whereE(HN) denotes the average hopcount between two arbitrary
nodes. Fork-regular graphs this gives the answer one would expect.
We wish to investigate the exponential growth of a graph also in terms of the diameter of the graph. The logarithmic growth of the diameter, which is the maximal hopcount between any two pairs of nodes in the graph, should be closely related to the exponential growth of the graph. A test case could be to study the diameter of the random graph with exponential weights. 6.4.2 Part II: Modeling the end-to-end delay
Motivation.
Today, Internet is based on best-effort service, meaning that there is no guarantee given on the maximum delay that a data-packet can experience when traveling from source to destination. This delay is already annoying for the individual user waiting for pages that his favorite search engine has found, but it can make future services such as voice over IP impossible, since, as is well known in telephony, a one way delay exceeding 150-200 ms makes ordinary conversation impossible.
It is our goal to give a thorough and accurate statistical analysis of the delay of probe-packets on a fixed path and to distinguish the different components that lead to this delay. One of the components is the queueing delay, which is measured in an indirect way through the delay of our probe-packets. Hence we deal with a practical example of indirect data.
Before we continue to describe how we will reach the above goal, we say a few words on how Internet traffic is modeled nowadays.
In a by now famous paper Leland et al [13] showed that Ethernet LAN traffic at Bellcore exhibits self-similar behavior and long range dependence(LRD). This behavior is in contrast with conventional traffic (Poisson modulated traffic) which is not self-similar and has an exponential decaying covariance. An explanation of the long range dependence was found in [22], where it was shown that at the source level the packet trains can be modeled as on/off process, and that especially the off-periods have lengths with statistical heavy tails. This effect causes the LRD. Moreover it was shown that the sum of on/off processes, properly scaled, converges to fractional Brownian motion, which is a self-similar process and exhibits LRD.
We are not the first ones to model end-to-end delay in Internet: [3], [9] and [19] report and analyse end-to-end delay measurements. These measurements are mostly based on round trip times. The measurements are limited by a lack of clock synchronization, and asymmetries in
the one way and return paths. To surmount these problems researchers ([1], [7] and [4]) used multicast inference which can be an alternative if sufficient accurate one way data is not available. The RIPE measurements do not possess these drawbacks and are therefore unique. Moreover the RIPE data also contain the trace routes so that path variations can be singled out.
Proposal.
We propose to analyse the RIPE delay measurements in the following way. Since these mea-surements contain the delays on one-way trips together with trace-routes, we can analyse the one way delay of all packets between two boxes that take an identical path (the majority of probe-packets during one day stick to one fixed path). Our plan is to split the delays over a fixed path in two or more time periods, depending on the hour of the day (it is well known that delays have a diurnal rythm). The discriminating factor can be the hourly standard deviation, which varies from hour to hour depending on how much Internet traffic is present on the studied fixed path.
Most off the delay-histograms of the RIPE data resemble a Gamma-shape histogram with a long or even heavy tail. During the quiet hours (for two boxes in the same time zone mostly during the night) the stochastic delay (the part of the delay caused by queueing and processing) is mainly caused by processing time in the routers and only modestly suffers from Internet traffic on the path. During the busy hours on top of this processing delay there is a clear and substantial additional delay caused by Internet traffic.
The processing delay caused by the routers can be modeled conventionally. In lab-measurements, performed by two of our students [5], it was shown that the router delay of a single router has a symmetric density, with exponential tails. This behavior is confirmed in [17]. Since on a fixed path the number of routers is known and in between 5 and 25, the total processing delay can presumably be well approximated by a Gaussian.
Following [22], we plan to model the delay caused by Internet as a superposition of on/off processes, so that the total stochastic delay is a convolution of the delay caused by the Internet and a Gaussian density. More precisely our research in this topic covers:
End to end delay of a single path.
Our first task will be a statistical analysis of the delay caused by Internet traffic, which as described up to here comes to us in the form of indirect data. So we will focus on deconvolving the data using parametric or non-parametric statistical methods. This yields themarginaldensity of the delay on a fixed path.
Variations over time.
The delays described above will be dependent on the day of the week and (since measurements of the last two years are available) will give some picture of the growth of the Internet over time. It is important to model this behavior over time and we will also address this question. Possibly we can deduce from this behavior whether the Internet grows exponentially in time.
Path correlations.
A challenging problem is to model the interaction between the delays. Data can again be obtained from the RIPE measurements by making a map of the 40 boxes and the paths between these boxes. We then obtain data of a network, which gives insight in the larger Internet.
Jitter.
The jitter or delay variation is an important measure for real-time services, in particular for telephony because the human ear is very sensitive to jitter. The interest is in modeling the distribution of the maximum delay variation. The maximum jitter is one of the quality of
service (QoS) parameter and a user can ask the network a certain stringency on the maximum jitter of his application/service. The behavior (distribution) of the ’worst case’ jitter is therefore of importance to provide guarantees on this QoS parameter. Jitter is measured as followed. The source sends two packets back to back and the receiver measures the difference between the arrival time of these two consecutive packets. This measurement technique also provides information about the capacity of the path from source to destination. By complementing this measurement technique by another one, namely sending a train of packets with increasing length, additional information about the capacity of the path can be obtained. Finally, a correlation of these measurements over several overlapping paths, may (at least in principle) allow to deduce the capacity of individual links. This knowledge may lead to a better modeling of the link weight distribution in Internet (for which particular distributions have been assumed in part I).
7
Work programme
The work programme of the two requested Ph.D. students will be the following. The first column describes the research topics of the PhD student in Probability and Statistics, the second the PhD student in Electrical Engineering. We expect these students to communicate frequently, so that both can gain from the other’s expertise.
First year literature literature
familiarity with measurements familiarity with measurements hierarchical model end-to-end delay(1 path) Second year power laws/multicast test Variation over time
additional measurements? additional measurements? adaptation of model adaptation of model Third year exponential growth path correlation and jitter Fourth year writing of thesis writing of thesis
The postdoc will cooperate in this scheme with an emphasis on modeling. He/she will start at the middle of the first year until the end of the third year.
8
Requested budget
2 Ph.D students each for 4 years: 259,758 euro. Postdoc for 2.5 years: 126,494 euro.
Total: 386,252 euro.
Requested material budget during total project period in euros: computers travel other costs 6000 incl. in benchfee
-References
[1] A. Adams et al, The Use of End-to-end Multicast Measurements for Characterizing
In-ternal Network Behavior, IEEE Communications, 38, no. 5, pp 152-159, 2000.
[2] F. Begtasevi´c and P. Van Mieghem, Measurements of the
Hop-count in Internet, Proceedings of Passive and Active Measurement (PAM2001), Amsterdam, The Netherlands, April 23-24, pp. 183-190, 2001. http://wwwtvs.et.tudelft.nl/people/piet/teleconference.html
[3] J.C. Bolot, Characterizing End-to-end Packet Delay and Loss in the Internet, J. High
Speed Networks, 2, no 3, pp 289–298, 1993.
[4] R. C´aceres, N.G. Duffield, J. Horowitz, and D. Towsley, Multicast-based
Infer-ence of Network Internal Loss Characteristics, To appear in IEEE/ACM Transactions on Networking.
[5] C.J. Bovy, H.T. Mertodimedjo, G. Hooghiemstra, H. Uijterwaal, and P. Van Mieghem, Analysis of End-to-end Delay Measurements in Internet, Accepted by PAM
2002. http://wwwtvs.et.tudelft.nl/people/piet/teleconference.html
[6] R.P. Dobrow and R.T. Smythe, Poisson Approximations for Functionals of Random
Trees, Random Structures and Algorithms,9 (1996), 79-92.
[7] N.G. Duffield, and P.F. Lo Presto, Multicast Inference of Packet Delay Variance at
Interior Network Links, IEEE Infocom 2000, Tel Aviv Israel, 2000.
[8] M. Faloutsos, P. Faloutsos and C. Faloutsos, On Power Law Relations of the Internet Topology, Poceedings of ACM SIGCOM’99, Cambridge Massachusetts, 251-262, 1999.
[9] A. Fei, G. Pei, R. Liu, and L. Zhang, Measurements on Delay and Hop-count of the Internet, Proc. of the IEEE Global Internet, December 1998.
[10] F. Georgatos, F. Gruber, D. Karrenberg, M. Santcroos, A. Susanj, H. Uij-terwaal and R. Wilhem,Providing Active Measurements as a Regular Service for ISP’s,
Proceedings of Passive and Active Measurement (PAM2001), Amsterdam, The Netherlands, April 23-24, pp. 45-56, 2001.
[11] R. van der Hofstad, G. Hooghiemstra and P. van Mieghem, First Passage
Per-colation on the Random Graph,Probability in the Engineering and Informational Sciences, 15, pp 225-237, 2001.
[12] R. van der Hofstad, G. Hooghiemstra and P. van Mieghem, On the
Covari-ance of the Level Sizes in Random Recursive Trees, To appear in Random Structures and Algorithms, http://ssor.twi.tudelft.nl/ gerardh/
[13] W.E. Leland, M.S. Taqqu, W. Willinger, and D.V. Wilson, On the Self-Similar
Nature of Ethernet traffic(extended version), IEEE/ACM Transactions on Networking 2, pp 1-15, 1994.
[14] P. van Mieghem, G. Hooghiemstra and R. van der Hofstad, A Scaling Law
for the Hopcount in Internet, Report 2000125, Delft University of Technology, Delft, the Netherlands.
http://wwwtvs.et.tudelft.nl/people/piet/teleconference.html
[15] P. van Mieghem, G. Hooghiemstra and R. van der Hofstad, Stochastic Model
for the Number of Traversed Routers in the Internet, Proceedings of Passive and Active Measurement (PAM2001), Amsterdam, The Netherlands, April 23-24, pp. 190-194, 2001. [16] P. van Mieghem, G. Hooghiemstra and R. van der Hofstad, On the Efficiency
of Multicast, To appear in IEEE/ACM Trans. on Networking,9, no. 6, December 2001 [17] K. Papagiannaki, S. Moon, C. Fraleigh, P. Thiran, F. Tobagi, and C. Diot,
Analysis of Measured Single-Hop Delay from an Operational Backbone Network, submitted to IEEE Infocom 2002.
[18] V. Paxson, End-to-end Routing Behaviour in the Internet, IEEE/ACM Transactions on
Networking,5, no. 5, pp 601-615, 1997.
[19] V. Paxson, Measurements and Analysisi of End-to-end Internet Dynamics, Ph.D. Disser-tation, University of California at Berkeley, 1997.
[20] G. Phillips, S. Shenker and H. Tangmunarunkit,Scaling of Multicast Trees:
Com-ments on the Chuang-Sirbu Scaling Law, Proc. ACM Sigcomm 1999.
[21] S. VanHastel, B. Duysburgh, and P. DeMeester, Performance Measurements on
the Current Internet, 7th IFIP ATM&IP Workshop, Antwerp(Belgium), June 28-30, 1999. [22] W. Willinger, M.S. Taqqu, R. Sherman, and D.V. Wilson, Self-Similar through
high-variability: Analysis of Etherenet Lan Traffic at the source level,IEEE/ACM Transac-tions on Networking 5, no. 1, pp 71-86, 1997.
