Peer-Assisted Content Distribution With Random Linear Network Coding. Worksheets

(1)

Peer-Assisted Content Distribution With Random

Linear Network Coding

Worksheets

Group 09gr720

Martin Hundebøll Jeppe Ledet-Pedersen

Georg Sluyterman

(2)

(3)

Chapter 1 Project Motivation

Network Coding is an interesting field of research and has, since [Ahlswede 00] introduced linear coding of packages for multicast purposes, been subject to increasing research.

Current research in network coding has primary had focus on multicast in wireless applications, where network coding benefits from changing packet losses on each nodes [Fragouli 06]. File sharing with network coding has had less attention and prior research disagree on whether network coding has an advantage in peer-to-peer file sharing compared to todays routed file sharing.

[Chiu 06] concludes that when investigating throughput alone, network coding has no advantage. However, [Chiu 06] propose that network coding performances better in dynamic networks, in which peers join and leave at arbitrary rates. In [Jain 05] robustness is taken into consideration when setting up network topologies for use with network coding. It concludes, that using network coding can help simplifying the design of the network.

1.1 Goals of this Project

Several papers (i.e. [Zeng 08], [Si 09], [Niu 07]) uses simulations to investigate the performance of network coding in peer-to-peer file sharing. When reading these papers, it is unclear if the results takes limitations in computational power, memory, etc. into consideration and thus the practical use of network coding is of interest.

This project is originally instpired by the work in [Yang 08] and [Gkantsidis 05], which describes a practical implementation of a peer-to-peer network protocol applicable to network coding. The protocol is proved useful in simulations, and thus a practical implementation of the protocol is desirable.

Using the algorithms for network coding developed in [Pedersen 09], a functional application is to be developed, so that the practical use of network coding can be tested. When developing the application, the protocol described in [Yang 08] is used as reference.

When testing the developed peer-to-peer application, the results are compared to an alternative file-sharing applications.

(6)

Chapter 2 Network Coding

As described in Chapter 1, network coding algorithms from [Pedersen 09] is used in the implemen-tation of the data exchange protocol. This chapter describes the underlying linear algebra and the required algorithms in detail. All matrix operations are performed in a Galois Field of order two, GF(2), containing only the elements 0 and 1. Vectors in the column space of a matrix can thus be formed by combining a subset of the columns with theXORoperation.

The network coding is performed in three different steps: encoding, recoding and decoding. The server and all clients who have obtained the complete file constructs encodes packets by creating linear combinations of partitions of the original data. The encoded packets are transmitted with the corresponding encoding vectors to connected nodes. Intermediate peers who have received a number of these packets, can form recoded packets by combining encoded data. When enough linear independent packets and encoding vectors have been received, the original data can be reconstructed by decoding the encoded packets. The algorithm and mathematical foundatation for each step is described in the following sections.

The network coding scheme uses Random Linear Network Coding (RLNC), in which the encoding vectors are generated randomly, in contrast to Deterministic Linear Network Coding. This introduces the possibility to generate linearly dependent packets, i.e. packets containing only redundant information. The likelihood of generating packets containing non-usefull information depends on the selected coding parameters. Due to these redundant packets, the total amount of data transferred between peers can thus be larger than the size of the original file.

In order to encode the original data, the file is divided into a number of equally sized packets. If the file size does not divide equally into a number of packets, the last packet is zero-padded. The file is further subdivided into a number ofgenerations each containing an equal number of packets denoted the generation size. Note that the final generation may contain fewer packets than the other. This is illustrated in Figure 2.1 which shows a file containing 23 packets and 5 generations. The generation size is 5.

Figure 2.1: Division of a file containing 5 generation and 23 packets.

In the description of the algorithm, the following notation is used: • bis the number of bits per packet.

• gis the number of packets per generation. • ris the number of redundant packets.

(7)

2.1. ENCODING

• iis the number of missing packets needed to decode the generation.

The number of generations n, needed to encode a file of sizesbits can be calculated by: n= s b·g (2.1)

2.1 Encoding

The data is encoded by organizing each generation in a matrixMof sizeb×g, where each column contains the bits of an uncoded packet and the number of columns depends of the generation size. To generate a single encoded packet,x, a coding vectorgis generated withg random bits [g1, g2, . . . , gg]T. The coding vector is multiplied with M to generate x. To code an entire

generation with redundant packets, the matrixGis built withg+r coding vectors and multiplied withMyieldingX. Therredundant packets are needed to account for possible linearly dependent encoding vectors.

Encoding Packets

Data matrix and coding vector and matrix:

Mb×g= [m1,m2, . . . , mg] (2.2)

g= [g1, g2, . . . , gg]T (2.3)

Gg×(g+r)= [g1,g2, . . . , gg,gg+1, . . . , gg+r] (2.4)

One coded packet:

x=M×g (2.5)

g+rcoded packets:

X=M×G (2.6)

2.2 Recoding

When only some of the packets needed to decode the data are available, recoding is possible. This is used to ‘share’ data between peers, so that two peers holding different parts of a generation, might be able to decode the generation by exchanging data. Recoding is done by multiplying both the matrix containing received coded packetsXb and the received coding matrixGb with a new coding matrixHto generateXe andGe. Due to the linearityXe will still contain linear combinations of the original data.

Recoding Packets Partially received data:

b

Xb×(g−i) (2.7)

Partially received received coding matrix: b

Gg×(g−i) (2.8)

Recoding vector:

(8)

CHAPTER 2. NETWORK CODING

Recoding matrix:

G(g−i)×k (2.10)

Where k is the number of packets required to obtaing lineary independend packets. The value of k is not known by the implementation, since it depends on lineary independence and how many packets other peers request. One recoded data packet:

e

x=Xb ×h (2.11)

One recoded coding vector:

e g=Gb ×h (2.12) (g−1) +rrecoded packets: e X=Gb ×H (2.13)

2.3 Decoding

When the received coding matrixGb containsg or more linearly independent coding vectors, i.e. rank(Gb) is equal tog, the generation can be decoded. This can be performed mathematically by multiplying the received coded data matrixXb with the inverted received coding matrixGb−1. The requirement on the rank ofGis exactly to ensure invertibility of the matrix. If the data is transmitted using a perfect link,Xb =XandGb =G

Decoding Packets Received data:

b

Xb×g (2.14)

Received coding matrix:

b

Gg×g (2.15)

Decoded packet:

b

X×Gb−1=M (2.16)

which requiresGb to be invertible: rank( ˆG) =g, whereg is the no. of linear independent columns.

Decoding in Implementation

In the implementation, the decoding is done by constructing an augmented matrix containing the matricesGbTandXbT. The Gauss-Jordan algorithm is applied to the rows of the augmented matrix until the left side contains the identity matrix. The righthand side of the augmented matrix now contains the original data.

b GT|XbT =      b g1 b g2 .. . b gg b x1 b x2 .. . b xg      ⇓ I|cMT =      1 1 . ._. 1 b m1 b m2 .. . b mg     

(9)

2.4. CODING EXAMPLE

2.4 Coding Example

To illustrate how the coding works in practice, a small example with a single generation of nine bits is given here. The original data is organized in a 3×3 matrix and a coding matrix is generated randomly.

Encoding the Data

M=   0 1 0 1 1 0 1 0 1   G=   1 0 1 0 0 1 1 1 1   X=M×G=   0 0 1 1 0 0 0 1 0   (2.17)

Decoding the Data

b GT|XbT =   1 0 1 0 0 1 1 1 1 0 1 0 0 0 1 1 0 0   (2.18) ⇓ I|McT =   1 1 1 0 1 1 1 1 0 0 0 1   (2.19)

2.5 Choosing Parameters

(10)

Chapter 3 Protocol First Draft

The protocol described [Yang 08] is used as reference for the implementation of the file sharing application, but some specifications are changed to suit the network coding algorithms developed in [Pedersen 09].

3.1 Network Topology

To give an understanding of the decisions made in the protocol description, an overview of the network topology is given here. The goal is to support the network coding (and decoding) described in Chapter 2, that is packet recoding on the peers for further transmitting.

To achieve as much different linear combinations as possible, while keeping the protocol simple, the peers are divided into groups corresponding to the number of linear combinations needed to decode the file, added extra groups to achieve redundancy.

Each peer establishes a yet to be determined number of connections to peers in the same group (intra-neighbours), and another number of connections to peers in other groups (inter-neighbours). When receiving data from peers in other groups, the peer recodes this into a package and pushes this to peers in its own group.

The topology is illustrated in Figure 3.1, in which node 1 receives coded packages from its inter-and intra-neighbours The peer recodes the received data into new coded packages inter-and pushes it to its inter- and intra-neighbours. When peer 2 receives the coded package from peer 1, it follows the same procedure and pushes recoded packages to its own neighbours, who then again recodes the packages and pushes it forward.

Figure 3.1: The topology of the overlay network.

The following conditions apply to the overlay network:

(11)

3.2. PEER JOINING

• When assigning peers to groups, the server tries to balance the group sizes. • A peer holdsxlocal connections.

• The same peer holdsy remote connections. • Groups are sized in relation toxand y.

By applying the above rules, the distance to relay nodes and thus the server, is kept short.

3.2 Peer Joining

First event occurring after the server is started, is a peer joining. The sequence of events is illustrated in Figure 3.2.

Figure 3.2: Events occurring when a new peer joins the network.

The peer sends a JOIN package to the server, who assigns the peer to a group. If the joining peer is the first in a group, the peer is selected as relay peer. Should the peer be the first peer joining the network, it is selected as relay peer in all groups. This ensures that even with only one peer, the file is downloaded. When new peers join, the first peer is replaced as relay peer in the group of the joining peer.

To optimize network utilization, a joining peer may replace the existing relay peer, if the joining peer has a better connection. How to measure the connection is to be determined.

Two lists are transmitted to the joining peer: one with addresses of peers in the assigned group and one with addresses of a peer in each other group. The joining peer connects to a number of peers in the assigned group and to one peer in a number of other groups.

3.3 Peer Leaving

When peer decides to leave the network, the other peers connected to the leaving peer must be notified in order to establish alternative connections. The events occurring when a peer leaves gracefully is illustrated in Figure 3.3.

3.4 Peer Crashes

If a peer is disconnected without notifying the peers connected to the disconnected peer, a timeout occurs on the periodic HELLO commands. When this happens, alternative connections must be established and the server must be notified. The process is illustrated in Figure 3.4.

When the neighbours encounter a time-out and the server has been notified, the neighbours of the disconnected peer initiates the rejoin process described in Section 3.3.

3.5 Data Transmission

The server definesngroups, corresponding to the number of generated combinations described in Section 2. As each group is ensured to hold a peer (if the number of peers is less thann, the same peer might peer occur in multiple groups), the server can start transmitting encoded packages.

(12)

CHAPTER 3. PROTOCOL FIRST DRAFT

(a) The peer decides to leave and sends LEAVE to con-nected peers and the server.

(b) The disconnected peers rejoins the server and the disconnected inter-neighbour request a new list of inter-neighbours.

(c) The server sends a list of peers in the group and the disconnected inter-neighbour receives a list.

(d) The needed connections are established.

Figure 3.3: Procedure to reestablish connectivity when a peer leaves.

(a) The neighbours to the crashed peer encounter time-out on the HELLO message.

(b) The inter-neighbour with the nearest group-ID above the crashed peer notifies the server.

Figure 3.4: How the server is notified when a peer disconnects without sending LEAVE messages.

(13)

3.6. PRELIMINARY CONCLUSION

The relay peer receives the package and pushes it to peers who initiated the connection to the relay peer. The next receiving peer pushes the package to the peers who initiated a connection to the current peer. Since peers only hold one connection to one peer in the same group, multiple transmissions of the same package to one peer is avoided.

Figure 3.5: The server pushes a package to the relay peer in group 1, who pushes the package to connected peers, etc.

3.5.1 Redundancy

The random network coding described in Section 2 introduces a probability that the received encoded packages are linear dependent and thus contains only redundant information. Thus an overhead is introduced to minimize this probability.

If each peer needs g encoded packages to decode the original file, it must be connected to one intra-neighbour andg−1 inter-neighbours.

3.6 Preliminary Conclusion

While considering how the described protocol is to be implemented, several questions were raised: • How is the overlay network initialized?

• How should the overlay network be maintained without to much overhead and complexity? • How is data distributed to all peers?

• Are the computational requirements while coding to high? In the following, the raised questions are discussed and answered.

3.6.1 Initialization of the Overlay Network

It is assumed, that peers joins the network continually with a regular rate. Thus some rules is needed to initialize the network in a prober way. When the first joins the network, multiple options are available: Place the single peer in all groups (Figure 3.6(a)) or create only one group (Figure 3.6(b)).

When assigning the peer to all groups, later joining peers should replace the prior joined peers in some groups. Again multiple options are available: Replace the prior joining peer in half the groups or in only one group. If one chooses to replace in only group, the peers in multiple groups are more loaded than peers in only one group. Contrary, if the new peer replaces half the groups, overhead is introduced in moving peers between groups.

The second option, assigning the peer to only one group, other choices are to be made: Should new peers fill one group before starting the next or should new peers be assigned to empty groups? If one group is filled with peers, what happens when the next peer joins and is assigned to a new

(14)

CHAPTER 3. PROTOCOL FIRST DRAFT

(a) Placing the first peer in all groups. (b) Placing the first peer in only one group.

Figure 3.6:Two of the possible assignment rules when the first peer joins the network.

group? When transmitting data as described in Section 3.5, the peers in the full group should connect to a inter-neighbour. Again, the peer in the new group is more loaded than the rest.

3.6.2 Maintaining the Overlay Network

As with the initialization, groups need some thoughts on maintenance: What should be done, when a group is almost empty? An option is to wait for joining peers and assign these to the group. This again leads to more load on the few peers left in the group.

Another option is to move peers from the groups containing highest number of peers. Also this introduces overhead in reassigning peers and making these connect to peers in the new group.

(15)

Chapter 4 BRONCO Protocol

After several considerations, the protocol design described in Chapter 3 is changed substantially. The principle of groups of peers is abandoned, as these introduced unnecessary overhead in keeping the group sizes balanced.

As an example, overhead is introduced when the first group is at max size and a new peer joins. Should the new peer be left alone in the new group or should some peers from the first group be reassigned to the new group? Each solution is insufficient, as having a group with only one peer bias the network and moving peers gives unnecessary disconnects.

Furthermore, similar overhead is seen when the majority of peers in a group leave. Should the remaining peers be reassigned to other groups or should peers from other groups be moved to the group in question? Both solutions yields unnecessary disconnects.

It is assumed, that peers frequently join and leave the network and thus the principle could stress the server and decrease data throughput.

4.1 Network Topology

The network topology forms a partially connected mesh network with randomly connected peers as illustrated in Figure 4.1. The purpose is to ensure that each peer has multiple paths to the server and thus making the network robust to peers leaving and joining frequently.

Once a peer is connected, the server acts as a regular peer, transmitting random coded packets. The peers exchanges data using random linear network coding as described in Chapter 2.

Figure 4.1: Network topology with random connected peers.

To keep the TCP overhead low, each peer is configured to a maximum number of connections. Every connection to other peers are used for both receiving and sending data. When joining the

(16)

CHAPTER 4. BRONCO PROTOCOL

network, the peer informs the server if it has the complete file and thus only seeds data. When connecting to other peers, it will skip peers also having the complete file.

The protocol is divided into two sections, one describing the instantiation and maintaining of the overlay network, and one describing the communication between peers.

4.2 Overlay Network Protocol

The protocol for creating the topology described in Section 4.1 is named the Overlay Network Protocol. A goal of the protocol is to keep as simple as possible, while supporting the properties of network coding.

4.2.1 Peer Joining

Peers join the network by contacting the server for information about the network. When contacting the server, the peer sends a list of informations:

• 20 byte hash ID of the requested file. • 20 byte hash ID of the peer.

• A boolean specifying if the peer has the complete file.

• The local port on which the client is listening for other peers. • The number of peers to return to the joining peer.

The joining peer sends only the listening port and not the listening address to the server. The server reads the address from the join session, as the joining peer might be behind NAT and thus not know its global address.

The number of requested peers should generally be larger than the maximum number of connections the client can accept. The server may reply with peers which are unable to accept more connections or peers that have non-gracefully left the network and thus the peer needs spare peers to connect to.

The server completes the handshake by sending a list of informations about the file and the requested number of peers. The file informations are:

• Name of the file • Size of the file • Generation size • Number of chunks

The list of peers is generated by randomly selecting peers from its list of peers in the network. The list contains the following information about each peer:

• 20 byte hash ID of the peer. • Address of the peer.

• Port number the peer is listening on.

(17)

4.3. DATA FLOW PROTOCOL

Figure 4.2: Events occurring when a peer joins the network.

Figure 4.3: A leaving peer (1) sends leave commands to its connected peers and the server

4.2.2 Peer Leaving

Peers leaves the network in one of the possible ways: Gracefully leave and non-gracefully leave. When leaving gracefully, connected peers and the server are informed of the exit before leaving, by using the data flow protocol, as illustrated in figure 4.3. Non-gracefully leaves occurs when the peer crashes or is disconnected from the network, as illustrated in figure 4.4.

If a peer detects a connection failure to another peer, it reports this to the server by sending a report command including the hash ID of the disconnected peer. The server notes the report in the list of peers and when three number of disconnects are reported, the peer is removed from the list. By requiring three reports, a single peer cannot wrongly remove another peer (e.g. if the problem is lying at the internet connection for the reporting peer).

Figure 4.4: A peer leaves non-gracefully and the affected peers report to the server.

If the number of connections on a single peer is decreased, it should try to connect to spare peers received from the server. When out of spare peers, the peer may request more peers from the server.

4.3 Data Flow Protocol

When peers interconnect, handshakes are performed before transmitting linear combinations. After handshaking, a list of commands are available to the peers:

• Start • Stop

(18)

CHAPTER 4. BRONCO PROTOCOL

• Complete • Close

4.3.1 Handshake

The handshake contains the following information from the connecting peer: • 20 byte hash ID of the peer.

• 20 byte hash ID of the file.

• Boolean specifying if the peer has the complete file.

The accepting peer responds an OKif it accepts the connection or a Busy if it has reached its maximum connections. Afterwards it sends its hash ID to confirm its identity and a boolean specifying if it has the complete file.

When the handshake is performed, the connection is symmetric and data is used for sending data both ways. Should the connection attempt fail (not in case of a busy peer), the connecting peer reports a non-gracefully leave to the server.

4.3.2 Data Transmission

When a peer is ready to receive data, it sends aStartcommand and the connected peer starts transmitting linear combinations, generated from its own decoded data. When the receiving peer obtains enough data to decode the complete file, it notifies all connected peer with a Complete command and the connected peers stops transmitting. Similarly, the server is notified with a Complete command. If two connected peers are complete, they close the connection using the Close command. The sequence of events is illustrated in Figure 4.5.

Figure 4.5: Sequence of events between peers for transmitting data.

When disconnecting to other completed peers, new connections are established to spare peers. If no spare peers are available, additional peers are requested from the server.

TheStopcommand is used to stop data transmission from a connected peer without closing the connection. This is relevant, if a peer changes connected peers to obtain better throughput, while keep sending data to the stopped peer.

(19)

Chapter 5 Software Design

As described in Chapter 1, this project focuses on testing the protocol in a environment as close to real life as possible. Thus an implementation of the protocol is needed, which is described in this chapter.

The application is developed in a agile style, where the design is developed in steps along with the implementation.

5.1 Deployment

The application is structured in two parts as shown in Figure 5.1, where only relevant classes are illustrated.

Figure 5.1: Deployment of the protocol application.

When Server is started server is instantiated to listen for client connections and peer is instantiated to listen for peer connections.

To join the network,clientinClient connects toserver inServer and receives a list of peers. Then clientinstantiatespeer, which connects to the received peers.

5.2 Classes

The class diagram in Figure 5.2 illustrates the classes in the application. The client starts by instantiatingclientand the server starts by instantiatingserver. Each class is shortly described in the following.

(20)

CHAPTER 5. SOFTWARE DESIGN

Figure 5.2: Classes used in the software.

5.2.1 Class client

The client opens a connection to the server as illustrated in Figure 5.1 and receives a list of peers. peeris instantiated and passed the received list of peers.

5.2.2 Class server

The server listens for connections and sends a list of peers on request. To be able to transmit coded data to other peers, a peerobject is instantiated and the server adds itself to the list of peers in the network. For each incoming connection from clients, an instance of serverconnis made.

5.2.3 Class protocol

Data is transmitted as an object implemented in protocol. The implementation describes how the data is serialized and the type of data in the object.

5.2.4 Class arguments

When starting either the client or the server, arguments passed to the application is parsed by arguments. The arguments includes server port, peer port, file to transmit, listening address etc.

5.2.5 Class serverconn

When incoming connections from clients are accepted, an instance of this class is associated with the connection. Data received on the connection is passed to an appropriate handler based on the data type. This class inherits from connection.

5.2.6 Class peer

This class listen for incoming connections from other peers and opens connections to peers received from the server. For each inbound or outbound connection, an instance of clientconnis made.

5.2.7 Class clientconn

As serverconn, this class is inherits from connectionand is associated with connections to the server or other peers. The class handles received data by passing it to a handler based on the received data type.

(21)

5.3. EXTERNAL LIBRARIES

5.2.8 Class connection

connectionis a base class for classes handling connections. It provides methods for reading and sending data. When sending data, the data is serialized and written to the socket. When data is read from the socket, it is deserialized and passed to the handler.

5.2.9 Class coder

When connected to other peers,coderproduces encoded packets for transmitting and consumes received encoded packet for decoding.

5.3 External Libraries

The implementation depends on several external libraries1_:

boost::asio boost::thread boost::serialization boost::shared_ptr boost::program_options Crypto++ Gf2

(22)

Chapter 6 Test Descriptions

6.1 Network Model

Due to problems with TC in Linux, we needed to drop the extended star-like topology of networks and instead of have a star topology where all networks are connected to the same high-speed backbone. That means that the description in subsection 6.1.2 was not implemented in the final test setup. Furthermore variance with some distribution is not supported in Dummynet. For further details see section 6.3

When the application has to be tested, it is desireable to test the application in an enviro-ment which is as close to a real world scenario as possible, and preferably at the same time perform a test which can be reproduced. Defining what a “real world” scenario is is a complex task, and [Floyd 01] state that there exist no single suite of simulation scenarios that can test a protocol for its behavior in the future evolving internet. And since the internet is rapidly ever evolving, testing a protocol in a test enviroment does not make it easyer. Another issue is the size of a given test setup. The real internet contains millions of nodes, routers and so on. A number and complexity we can not get near in a laboraty. The special case errors that may appear a number of times on the real internet because of its size and diversity may happen very seldom if even at all in the laboraty. Our focus on our test in the lab will therefore be based on some more basic attributes. To get closer on how the application would perform on the internet it could be argued that getting random users on the internet to download the application and test it would get us close to a “real world” scenario. But would the users that download the application for testing in any way represent another set of users actually using the application, if it were released into the public. Getting random real life users to test the application on the internet also makes it very hard to reproduce our test, since we do not have a well definet test enviroment, and without a somewhat well defined test enviroment it is also hard to measure what effect the change of a given variable would have on the application if the enviroment has changed in the mean time.

6.1.1 Network Attributes

6.1.2 Emulation

Since we can not build a representive copy of the internet i our lab, we have setup a network for emulation or make a simulation. A simulation of a protocol that is based on network coding has been made by [Yang 08]. Our focus is to build an application that can be used and tested on a real network. This also makes it possible to work upon the work we have made, with regard to an application for real life use. Although a simulation can introduce a lot of network attributes it is still a simulation with the shortcommings it has. We will therefore produce an application written for use on real IP networks on real life computers. We will in the following go through the basic

(23)

6.1. NETWORK MODEL

Percentage Downstream k[bps] Upstream k[bps]

10 256 256 20 2048 256 40 4096 512 10 10240 1024 15 10240 4096 5 20480 10240

Table 6.1: Distribution of bandwidth on connections in Denmark

parameters and attributes that we set up for the test network. The test network

We believe the most obvious attributes for the end users internet connections to be: Speed, latency, packet loss and basic network topology. For simplicity we neglect packet loss which is normally not a big factor on wired connections (but may be on wireless connections). Never the less since our application work on the application layer, we do not see the packet loss directly, and it would just map a degree of latency because of the connection orioented nature of TCP.

The individual speed for the nodes have in the test been replaced with cwidentical bandwidth symmetrical lines with 1, 5 or 10 Mbps, depending on the test being run. With fixed lines instead of lines with different bandwidths it is more straight forward to deduce something from the results since the results are more clear and also gives a better statistical basis, since we then have 36 identical nodes on the network. If the bandwidths are individual for each node the length of a test will often depend on the slowest node in the network, making measuring the effenciency of a protocol or application much more complicated.

End User Network Connection Speed The network speed for a typical user can vary. One approach to find the composision of speeds of the population would be to go through the half year statistic analysis of internet connections in Denmark, computed by the danish National IT and Telecom Agency. However this is for Denmark only and does not state how the situation is in other countries. Also we have no knowledge about how the distribution of the users are across countries1, and in a country across the distribution of users across the different internet network connection speeds. E.g. it could be that that the density of users of a P2P network is higher in some countries than in others and that the users using the P2P network are generally having a more speedy connection. However to have som realistic picture of the distribution of the internet cennection speeds, we use the same distribution as the one in the latest half year statistic from the Danish National IT and Telecom Agency. For simplicity we will group the connection speeds into only 6 groups of speeds instead of the 9-11 groups that the Danish National IT and Telecom Agency use. The different speeds documented in table 6.1. For more informatio see Appendix A.

End User Network Connection Latency The danish National IT and Telecom Agency does not collect statistic about the delay on internetconnection. Delay can probably to some extend be guessed based on what latency we can mesure on our own ADSL-lines at home. We would like to try to introduce latency with some mean and some varians into the end users network connection. We wil base this on what we have measured on a few ADSL-lines with the help from the Unix command “traceroute”. Although we do not have any good statistic at hand, besides the following measurements, it should be sufficient enough since the internet it self is so diverse that we will probably not get much cloaser to the answer even with many measurements of danish internet connections.

1_{Statistics from OECD and the European Union does only mention number of connection etc.} _{Not the}

(24)

CHAPTER 6. TEST DESCRIPTIONS

IP address delay to ISP router 10−3 [s] 89.150.136.21 23.752 89.150.140.140 26.756 89.150.155.123 24.066 89.150.169.221 24.216 89.150.174.34 22.153 89.150.178.134 23.154 89.150.178.145 22.302 89.150.179.83 18.383 89.150.180.54 24.615 89.150.181.98 24.656 89.150.182.197 22.098 89.150.184.76 18.260 89.150.185.136 21.711 89.150.186.139 26.883 89.150.187.184 21.743 89.150.187.204 20.453 89.150.188.55 26.062 89.150.188.58 25.601 89.150.188.82 25.216 89.150.189.212 25.083

Table 6.2: Latency between last and second last hop from the Danish ISP Fullrate measured with the Unix command traceroute (ICMP) from munin.dotsrc.org

The results in table 6.2 have mean 23.358 and standard deviation 1.6260. Inspired by this the nodes connection to the network in the test setup will have a delay with the same mean and deviation (although only with two significant digits).

The ISP’s Network Connections

If we connect every user to a big backbone, which should act as a single network we may be making a too simplistic network to test our application on. The internet is a network of networks, and all ISP’s are not necessary connected very well to each other. E.g. a user in Africa may not be able to get a decent throughput to users outside of Africa, or a user may have better connectivity within an ISP’s own network. We therefore build another layer on top of every users connected to a single fast network. We let users be grouped together, connected to a single network. These networks are then connected together with a certain speed (and again for simplicty with a fixed latency) to a common network with high speed. See figure 6.1 for a principal drawing of the topology. The real internet is a big web of networks connected to each other with different line speeds and latency, and not some networks connected to a central network. We hower choose to make our network as an extend star toploogy as a tradeoff between complexity and the fact that some users on the internet (typically customers at the same ISP or inhappitens of the same country) can reach each other with better throughput. We have 36 computers to test the application on. We choose to divide the computers in to six subgroups, forming six subnets. Each of the 36 computers will be assigned a internet connection speed limit (see table 6.1) and then spread amongst the subgroups forming the subnets. We have no statistic to rely on with regard to what speed the connection from a subnet to the backbone should be. With all other parameters fixed, we will, depending on what we meassure, try to set the speed limit to the backbone to different percentages of total download capacity in a given subnetwork. The connection from a subnetwork to the backbone will be symmettric. We choose to set the delay from the subnets to the backbone to different latencies between 20 m[s] (e.g. two naboring countries) and 120 m[s] (e.g. accros the atlantic). This means there will be the following delays: 20, 40, 60, 80, 100 and 120 m[s]. Since the network topoligy is a

(25)

6.2. ALTERNATIVE FILE SHARING APPLICATIONS

simplified model we believe this to be adequate.

Figure 6.1: Basic topology of the test network.

Test setup

Two computers (Intel P4 3.00 GHz HT, 2 GiB RAM) are used for infrastructure. One is acting used as a server (Debian GNU/Linux Lenny stable) with the server-application running, and the other (FreeBSD 7.2-RELEASE) is acting as a router, controlling bandwidth and latency between the networks with IPFW and Dummynet. The router is connected to a Cisco Catalyst 2900-series Gigabit Switch with a VLAN trunk (IEEE 802.1q). The 36 test-nodes on 6 different networks are located on seperate VLAN’s and the central server is located on another VLAN (see figure 6.2.

The 36 computers (running Debian GNU/Linux Lenny Stable) are controlled via SSH key based login from the central server. The bandwidth, latency and the standard deviation (normal distribution) for the latency on each of the 36 computers are controlled with TC in the Linux kernel.

6.2 Alternative File Sharing Applications

6.2.1 Testing the application

Above it is described how the test setup is organised with regards to the nodes and their connection to each other. It is now relevant to have a look at what we want to measure and how. The different measurements, transfertime, CPU utilization, link utilization, link overhead and % complete are described in differet paragraphs below. Besides what is being measured some parameters has to be adjusted: Generation size, packet size and the file size. The last parameter, file size, is also being used in measuring the time it takes to download with regular HTTP and BitTorrent. There can not be deduced much about the performance of our application with the comparison with BitTorrent, since we are only testing BitTorrent in one specific setup.

Measurements

Transfertime The time in seconds since epoch (1970-01-01T00:00:00) at the beginning of the transfer and when the complete file has been received.

The time is fetched in the shell by running the date command. All nodes are having their time synced with NTP from ntp.es.aau.dk.

(26)

CHAPTER 6. TEST DESCRIPTIONS

Figure 6.2: Overall test setup. Only the computers connected to VLAN 4 and 8 are drawn.

(27)

6.3. CHANGES IN THE TEST SETUP

CPU utilization We are interested in measuring the CPU usage with a sample frequency of 2 [Hz].

link utilization The speed by wich data is send and received is logged. This is measured with a frequency of 2 [Hz]

percentage complete The percentage by which the file is completely received. This is measured each time a packet is received.

Parameters

Having gone through what is going to be measured it is relevant to adjust some parameters between the tests. These are: Generation size, packet size and file size. Lets take origin in the file size. It is interesting to test with both small and large file sizes both becaurse there might be a tendency in e.g. the application performing very well after running for some time (hence large files is needed to be transfered) and becaurse it is a realistic use, since users both download small (e.g. an mp3 file or a small program), medium and large files (e.g. movies or a Linux distritution). The test will includ the file sizes 1, 10, 100, 1000 and 10000 MByte.

Description of generation size and packet size goes here.

6.3 Changes in the test setup

Due to difficulties with TC for bandwidth limiting with token bucket filter, it was necessary to change the network setup and let the FreeBSD router handle bandwidth limiting instead of. This has the disadvantage, that it is not possible to make the extended star-like topology interconnecting all the networks. Or in other words the “ISP”emulation (dividing the computers in to 6 subnets with different speeds and latencies for the backbone has been left out. Furthermore to make the tests more straightforward the bandwidth for each node is set to equal bandwidths.

6.3.1 New setup with IPFW and Dummynet

Every node, including the central server, is placed on a seperate VLAN, totalling 37 VLAN’s. With Dummynet it is possible to change the speed and latency for each VLAN for each direction. Unfortunately it is not possible to introduce variations in the delay. The net effect is the same as connecting all nodes together in the same network and adjusting the speed with TC on every node (that is if TC worked). The latency was previosly found to be 23 ms, but taking the latency between the overall “ISP” networks in to account the latency is set to 50 ms for all nodes.

(28)

Chapter 7 Tests

To evaluate the performance BRONCO, five tests are conducted on the test setup described in Chapter 6.

7.1 Comparison with Internet Emulation

To show how BRONCO performs in comparison to HTTP and BitTorrent (BT), a 10 MB file is download from the source using each protocol with the nodes being bandwidth limited to 5 Mbps. Each node is started up consequently with 1 second waiting time between them before the next startup.

the test is run five times for each protocol and the total transfer time is measured. For BRONCO, the generation size is set to 512 and the packet size is set to 6400 bytes.

7.1.1 Results

1 2 3 4 5 Mean Variance µ, 95% Conf. Int.

HTTP 316 315 315 315 316 315.4 0.3 (315.0, 315.8)

BT 74 98 72 77 96 83.4 157.8 (-112, 279)

BRONCO 59 59 60 59 59 59.2 0.2 (59.0, 59.4)

Table 7.1: Transfer time for download to all peers.

To be fair, BitTorrent uses approximately 20 seconds to initialize transfers. A single test transferring 100 MB data, showed that BitTorrent is slightly faster than BRONCO. (264 seconds for BitTorrent and 280 seconds for BRONCO.)

7.2 Scalability

To evaluate how BRONCO scales in comparison to HTTP, the number of simultaneously down-loading nodes is varied. The test is run five times for both HTTP, BitTorrent and BRONCO with respectively 1, 2, 4, 8, 16, 32, and 36 nodes.

As with the test in Section 7.1, BRONCO is configured with generation size 512 and packet size 6400 bytes.

(29)

7.3. NETWORK CODING PARAMETERS HTTP 1 2 3 4 5 t¯ S2 µ, α= 0.05 1 18 17 17 17 17 17.2 0.2 (17.0, 17.4) 2 18 18 18 18 18 18 0 (18, 18) 4 37 37 37 35 35 36.2 1.2 (34.7, 37.7) 8 70 71 70 65 70 69.2 5.7 (62.1, 76.7) 16 141 140 140 140 140 140.2 0.2 (140.0, 140.4) 36 316 315 315 315 316 315.4 0.3 (315.0, 315.8)

Table 7.2: Results from increasing number of nodes downloading with HTTP.

BRONCO 1 2 3 4 5 t¯ S2 _{µ, 95% Conf. Int.}

1 22.5 22 22.5 23 22.5 22.5 0.125 (22.3, 22.7) 2 24.5 25 25 25 24.5 24.8 0.075 (24.7, 24.9) 4 27.5 27.5 27 27 27.5 27.3 0.075 (27.2, 27.4) 8 37.5 31.5 31.5 33 32 33.1 6.425 (25.1, 41.1) 16 44 42 40 40.5 40 41.3 2.95 (37.6, 45.0) 36 59 59 60 59 59 59.2 0.2 (59.0, 59.4)

Table 7.3: Results from increasing number of nodes downloading with BRONCO.

7.2.1 Results

7.3 Network Coding Parameters

The parameters of network coding is described in Chapter 2. This test evaluates how changes in the generation size with different connection speeds affects the performance of BRONCO.

Generation sizes of 32, 128, 512 and 1024 is evaluated with connection speeds of 1 Mb/s, 5 Mb/s, and 10 Mb/s. Each setup is tested five times.

7.3.1 Results

g 1 2 3 4 5 ¯t S2 _{µ, 95% Conf. Int.} 32 264 272.5 256.5 250 260.5 260.7 70.575 (173.1, 348.3) 128 163 181.5 156.5 168 166 167 84.625 (61.9, 272.1) 512 193.5 183 175.5 173.5 200 185.1 130.925 (22.5, 347.7) 1024 203.5 178.4 186.5 174.5 205 189.58 108.94 (23.050, 356.11)

(30)

CHAPTER 7. TESTS g 1 2 3 4 5 ¯t S2 µ, 95% Conf. Int. 32 63.5 63.5 63 68 63 64.2 4.575 (58.5, 69.9) 128 54.5 56 57 57 54.5 55.8 1.575 (53.8, 57.8) 512 59 59 60 59.5 59.5 59.4 0.175 (59.2, 59.6) 1024 58.5 58.5 59 58.5 58.5 58.6 0.050 (58.538, 58.662)

Table 7.5: Total transfer times from varying generation size and 5 Mb/s connections.

g 1 2 3 4 5 ¯t S2 _{µ, 95% Conf. Int.}

32 47.5 48.5 48 47.5 49 43.833 109.57 (-71.150, 158.82) 128 45.5 46 46 46 46 45.900 0.050000 (45.838, 45.962) 512 47.5 47.5 47.5 47.5 47.5 47.5 0.0 (47.5, 47.5) 1024 47.5 47.5 47.5 47.5 47.5 47.5 0.0 (47.5, 47.5)

(31)

Appendix A

Calculation of Line Speeds

On figure A.1 the statistics for internet connection speeds in Denmark second half year of 2008 is summed up.

The third column “100/99*percentage” in the tables “Upstream” and “Downstream” is to take into account the unspecified lines (we are interested in being able to sum up to 100% and not 99%).

The fourth and fifth column in the tables ”Upstream” and ”Downstream” is the first attempt to simplify the number of connections. E.g. “256 kbps” includes both 144 kbps and 256 kbps. The sixth column in the tables “Upstream” and “Downstream” is a round-off of the numbers from coulmn five.

The seventh column in the tables “Upstream” and “Downstream” is called “Divideable by 36”. As we are interested in having relative few different connection speeds, we further round off the values from column six to a percentage divideable by 36 (since we have 36 nodes to test the application on), and move a few percentages, most notably from upstream from 4 Mbps to 512 kbps.

That way it is possible to put together the six lines mentioned in the table “Types of lines (divideable by 36)”. These six connection speed are pretty much representitive for connections in Denmark. The accuracy is not that important since the statistics change the whole time and the composistion og line speeds may very well look different in other countries. The 36 different speeds will be distributed randomly across the 36 nodes.

(32)

APPENDIX A. CALCULATION OF LINE SPEEDS

Downstream:

Minimum bandwidth Percentage 100/99*percentage Bandwidth (including blanks above) 100/99*percentage Roundoff Divideable by 36

144 kbps 1 1,01 256 kbps 9,6 9,7 256 kbps 10,71 11 11,11 2 Mbps 17,3 17,47 2 Mbps 17,47 17 19,45 4 Mbps 45 45,45 4 Mbps 45,45 45 44,45 8 Mbps 8,5 8,59 10 Mbps 11 11,11 10 Mbps 19,7 20 19,45 20 Mbps 5,8 5,86 20 Mbps 5,86 6 5,56 50 Mbps 0,3 0,3 50 Mbps 0,3 0 100 Mbps 0,5 0,51 100 Mbps 0,51 1 Unspecified 1 Total 100 100 100 100 100,02 Upstream:

Minimum bandwidth Percentage 100/99*percentage Bandwidth (including blanks above) 100/99*percentage Roundoff Divideable by 36

144 kbps 6,7 6,77 256 kbps 23,6 23,84 256 kbps 30,61 31 30,56 512 kbps 37,4 37,78 512 kbps 37,78 38 44,45 1 Mbps 13 13,13 1 Mbps 13,13 13 11,11 2 Mbps 5,7 5,76 4 Mbps 6,9 6,97 4 Mbps 12,73 13 8,33 8 Mbps 1,2 1,21 10 Mbps 2,6 2,63 10 Mbps 3,84 4 5,56 20 Mbps 1,2 1,21 20 Mbps 50 Mbps 0,2 0,2 50 Mbps 1,41 1 100 Mbps 0,5 0,51 100 Mbps 0,51 1 Unspecified 0,9 Total 99,9 100 101 100,01 Types of lines (divideable by 36)

Percentage Downstream [kbps] Upstream [kbps]

11,11 256 256 19,45 2048 256 44,45 4096 512 11,11 10240 1024 8,33 10240 4096 5,56 20240 10240

Figure A.1: Calculation of line speeds, based on the 2008 second half anual report from Danish IT and Telecom Agency [IT 09].

(33)

Bibliography

[Ahlswede 00] R. Ahlswede, Ning Cai, S.-Y.R. Li & R.W. Yeung. Network information flow. Information Theory, IEEE Transactions on, vol. 46, no. 4, pages 1204–1216, Jul 2000.

[Chiu 06] Dah Ming Chiu, R.W. Yeung, Jiaqing Huang & Bin Fan. Can Network Coding Help in P2P Networks? In Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks, 2006 4th International Symposium on, pages 1–5, April 2006. [Floyd 01] Sally Floyd & Vern Paxson. Difficulties in simulating the internet. IEEE/ACM

Trans. Netw., vol. 9, no. 4, pages 392–403, 2001.

[Fragouli 06] C. Fragouli, J. Widmer & J.-Y. le Boudec. On the Benefits of Network Coding for Wireless Applications. In Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks, 2006 4th International Symposium on, pages 1–6, April 2006. [Gkantsidis 05] C. Gkantsidis, P.R. Rodriguezet al. Network coding for large scale content

distri-bution. In IEEE INFOCOM, volume 4, page 2235. Citeseer, 2005.

[IT 09] Danish National IT & Telecom Agency.

Telestatis-tik 2. halv˚ar 2008. Internet, pdf, juni 2009.

http://www.itst.dk/statistik/Telestatistik/halvarsstatistik/2008/telestatistik-for-1-halvar-2008-1/resolveuid/4a0181b6e9bdca0bbde28ed859da5129.

[Jain 05] Kamal Jain, László Lovász & Philip A. Chou. Building scalable and robust peer-to-peer overlay networks for broadcasting using network coding. In PODC ’05: Proceedings of the twenty-fourth annual ACM symposium on Principles of

distributed computing, pages 51–59, New York, NY, USA, 2005. ACM.

[Niu 07] Di Niu & Baochun Li.On the Resilience-Complexity Tradeoff of Network Coding in Dynamic P2P Networks. In Quality of Service, 2007 Fifteenth IEEE International Workshop on, pages 38–46, June 2007.

[Pedersen 09] M.V. Pedersen, J. Heide, F.H.P.Fitzek & T. Larsen. PictureViewer - A Mobile Application using Network Coding. In European Wireless 2009, Aalborg, Denmark, may 2009.

[Si 09] Jingjing Si, Bojin Zhuang, Anni Cai & Yinbo Cheng. Layered Network Coding and Hierarchical Network Coding for Peer-to-Peer Streaming. Circuits, Communications and Systems, Pacific-Asia Conference on, vol. 0, pages 139–142, 2009.

[Yang 08] Min Yang & Yuanyuan Yang. Peer-to-Peer File Sharing Based on Network Coding. In ICDCS ’08: Proceedings of the 2008 The 28th International Conference on Distributed Computing Systems, pages 168–175, Washington, DC, USA, 2008. IEEE Computer Society.

(34)

BIBLIOGRAPHY

[Zeng 08] Hui Zeng, Jiaqing Huang, Shaoguo Tao & Wenqing Cheng. A simulation study on network coding parameters in P2P content distribution system. In Communications and Networking in China, 2008. ChinaCom 2008. Third International Conference on, pages 197–201, Aug. 2008.

Peer-Assisted Content Distribution With Random Linear Network Coding. Worksheets