Having described some key functionality of our deployment environment, we now give a brief overview of the components involved. These components together consist of about 5,000 lines of perl scripts.
1. batch.pl is the top-level script that starts the experiments on the deployment machine. This script lists a set of experiments. Each experiment is given an experiment id, which determines the directory name of where the logs will be placed. Each experiment also contains a path to the configuration file for that experiment. The configuration file lists the set of machines with their individually configured upload and download capacities. It may also contain some experiment-specific global options such as the name of target file, the
location of the BitTorrent tracker, and the port on which to run the applications.
The script calls the experiment.pl script for each experiment.
2. experiment.pl script runs everything pertaining to a given experiment. It regu-lates the setup, start and gather phases of each experiment. In the setup phase, using parallel ssh, it copies all of the software and configuration files to each machine. It may also copy the required target files to the seed machines that will be serving the file. It uses rsync to copy the target files and the software to save time, as typically the same target file is reused by the seeds in a number of consecutive experiments. It records the list of machines that succeeded in the white list, which it pushes out to the machines in the start phase. It deploys the tracker on a designated tracker machine. It also copies the wrapper.pl script to each machine that is used to stop and start software during the experiment.
The experiment.pl script also monitors the completion file on the tracker every 30 seconds. When it determines that the sufficient number of nodes have com-pleted the experiment, it terminates the software on the machines by invoking the wrapper.pl script on the remote machines and passes it a stop command.
After termination, it copies the logs, again in parallel, from the machines to a local directory named with the experiment’s id number.
3. wrapper.pl The wrapper script is run locally on each machine. It is run with dif-ferent arguments to kill old experiments, start new ones, clear the BitTorrent’s hint files and other directories. For dynamic experiments, where a client may be stopped and restarted multiple times during the experiment as in Section 6.3, the wrapper script remains running during the experiment to perform the stop and restart when necessary.
FairTorrent Evaluation
We implemented FairTorrent (FT) on top of both the original BitTorrent Python client, which implements the documented version of the BitTorrent protocol, and on top of the popular Azureus Java client. Each implementation was only several hundred lines of code, demonstrating that FairTorrent is simple to implement. We ran experiments on PlanetLab to compare FairTorrent in a realistic wide-area net-work environment against four other BitTorrent implementations: (1) original Bit-Torrent 3.9.1 (BT), the code base used for our FairBit-Torrent Python implementation, (2) Azureus 3.0.4.2 (AZ), the code base used for our FairTorrent Java implementation, (3) PropShare 1.1.1 (PS), the latest PropShare version available, and (4) BitTyrant 1.1.1 (TY) which users block-based TFT with other BitTyrant peers, (but acts strate-gically towards non-BitTyrant clients). To measure fairness and performance, each client logged its instantaneous maximum positive and negative service error (Emax+ , Emax− ), bytes uploaded and download from each peer, and its download completion time.
We present results for four sets of experiments: (1) clients with upload capacities across a uniform distribution all begin downloading simultaneously and remain as
seeds in the system upon download completions, (2) a high contributing peer partici-pating in a swarm with low contributors and free-riders, all clients begin downloading simultaneously and remain as seeds in the system upon download completions, (3) clients with upload capacities based on a distribution from live BitTorrent networks which join the system asynchronously, and leave upon download completion, and (4) live BitTorrent swarms. A 32 MB target file was used for all experiments except for the live BitTorrent swarms. Unless otherwise indicated, all clients were configured with their respective default number of simultaneous connections, 50 for Azureus and FairTorrent, up to 80 for BitTorrent, and 500 for PropShare and BitTyrant. Except for the live BitTorrent swarms, FairTorrent Java client and Python client results were similar, so only the Python client results are shown due to space constraints.
6.1 Uniform Distribution
The uniform distribution represents a wide range of peers with diverse upload capac-ities participating in a swarm. We used a network of 50 leechers and 10 seeds. We used a small seed fraction (0.2) to indicate that our results do not depend on high data availability. (An empirical study of live BitTorrent swarms [50] shows that on average the fraction of seeds is 0.5). All nodes were configured with download band-width capacities of 100 KB/s and upload bandband-width capacities of no more than 50 KB/s, reflecting typical scenario where most ISPs allow users a download rate at least twice their allowed upload rate. Each leecher received an upload capacity randomly selected between 1 and 50 KB/s. Seeds were configured with upload bandwidth ca-pacities of 25 KB/s each, chosen to match the average upload bandwidth capacity of the leechers, for a total of 250 KB/s of seed bandwidth. We ran five experiments with five different sets of upload capacities generated randomly from the respective
distribution, resulting in 250 leecher measurements. To first show the fairness and performance of each client in isolation, we ran the same set of experiments for a homo-geneous network of each client. PropShare turns off “seeding” capability by default.
To enable PropShare to work in a homogeneous network, we changed this default to allow its clients to “seed”.
Figures 6.1 to 6.21show the uniform distribution results. Figures 6.1 to6.5 show the average upload rate versus the average download rate from other leechers expe-rienced by each leecher during its download. A reference line is shown for the ideal case, in which the download rate should equal the upload rate. Figure6.1 shows that in FairTorrent the average download rate from other leechers closely matches each peer’s upload rate. Figures 6.2 to6.5 show that for BitTorrent, Azureus, PropShare, and BitTyrant peers’ download and upload rates do not match. Higher contribut-ing peers in BitTorrent, and even more so in PropShare and BitTyrant, are likely to receive a download rate far below their contribution, while lower contributing peers receive a higher level of service than their contribution. In Azureus, high contrib-utors are likely to deviate from their fair share. The non-FairTorrent systems are limited by their round-based rate allocation, where it may take many rounds for a high-contributing peer to find a more favorable set of peers to exchange data with.
In addition, all-BitTyrant network that defaults to a block-based TFT suffers from under-utilization for high uploaders due to hard limits on peerwise deficits. Figure6.1 shows that all high uploading FairTorrent leechers are resilient to low contributing peers and are able to obtain a matching leecher download rate.
Figures 6.6 and 6.7 show how quickly peers obtain rate convergence. Figure 6.6 plots a CDF for each system which demonstrates how quickly a fraction of peers converges to a download rate of at least 90% of their upload capacity. (e.g. how quickly does a 50KB/s peer begins to get at least 45 KB/s rate from other leechers).
Figure 6.6 specifically examines the high-uploading peers, those with capacities of 40-50KB/s. In FairTorrent 100% of high-uploading peers reach this convergence, as compared to 59%, 78%, 16% , and 62% for BitTorrent, Azureus, PropShare, and BitTyrant respectively. Moreover, in FairTorrent, due to its fast convergence property all the high-uploading peers reach convergence within 15 seconds from the start of the experiments. In other systems, it takes at least 800 seconds for the highest fraction of peers to reach convergence. Although, in theory PropShare algorithm exhibits fast convergence, it requires very long rounds to estimate all peer’s allocation rates.
In practice, with more reasonable rounds durations in PropShare, and with peers switching their peer active sets, the convergence process for high-uploaders is often interrupted and does not complete.
Figure 6.7 shows convergence for all peers. Including free-riders, 85% of peers reach convergence in FairTorrent, 73% (BT), 76% (AZ), 72% (PS) and 71% (TY).
The reason not all peers reach 90% convergence in FairTorrent is that for the low-contributing peers, the protocol overhead represents more than 10% of the upload capacity. Since the effective data upload rate is reduced to below 90% of upload capacity, the effective data download rate is also below 90%. (However, even low contributing peers obtain a matching effective download rate shown in figure 6.1).
For those peers that do converge in FairTorrent, the convergence is again very quick, with 82% of peers converging within 30 seconds. In contrast, it takes over 900 seconds, for the highest fraction of leechers to converge in other networks.
Figure6.8shows the maximum positive service error Emax+ and maximum negative service error Emax− for each of the four networks. For example, F T + and F T − denote Emax+ and Emax− of FairTorrent. A range of percentiles is shown for each network, starting with the 50th percentile, the median maximum service error across all leechers for a network, and up to the 100th percentile, the worst maximum service error.
0 10 20 30 40 50
0 10 20 30 40 50
Download Rate (KB/s)
Upload Rate (KB/s)
Figure 6.1: Uniform: FairTorrent fairness
FairTorrent has one to two orders of magnitude smaller error. The median FairTorrent Emax+ was just 79 KB. The maximum FairTorrent Emax+ was 436 KB, meaning that during the entire download of the file, no FairTorrent leecher gives more than 436 KB of service than what it receives from other leechers. This is 18 to 73 times smaller than the maximum Emax+ of other networks. The maximum Emax+ for Azureus and BitTorrent was over 8 MB, more than 25% of the entire 32 MB file. Emax+ for PropShare reached 19 MB, or 60% of the file. Emax+ for BitTyrant reached 31 MB, or almost 100% of the file. FairTorrent provides significantly better fairness than all other clients, which results in fast rate convergence and performance more correlated with a peer’s contribution.
Figure6.9shows the average and maximum download times for the high-uploading leechers in each network, those with 40-50 KB/s capacities. Figure 6.10 shows the download times of all the leechers. For the high-uploading leechers, the average download times were 690, 733, 745, 1027 and 963 seconds for FairTorrent, BitTorrent, Azureus, PropShare, and BitTyrant, respectively. The worst download times for the
0 10 20 30 40 50
0 10 20 30 40 50
Download Rate (KB/s)
Upload Rate (KB/s)
Figure 6.2: Uniform: BitTorrent fairness
0 10 20 30 40 50
0 10 20 30 40 50
Download Rate (KB/s)
Upload Rate (KB/s) Figure 6.3: Uniform: Azureus fairness
high-uploaders were 756, 876, 980, 1200, and 1298 seconds for FairTorrent, BitTorrent, Azureus, PropShare, and BitTyrant, respectively. Because of faster rate convergence and tighter maximum deficit, the FairTorrent high-uploading peers receive a more fair reciprocation for their contribution.
Figures 6.11 to6.15 show the completion times of all the leechers in each system
0 10 20 30 40 50
0 10 20 30 40 50
Download Rate (KB/s)
Upload Rate (KB/s)
Figure 6.4: Uniform: PropShare fairness
0 10 20 30 40 50
0 10 20 30 40 50
Download Rate (KB/s)
Upload Rate (KB/s)
Figure 6.5: Uniform: BitTyrant fairness
based on their upload capacities. Figure 6.11shows that FairTorrent provides a very strong correlation between upload capacity and download times. There is very little variability in download times among leechers of the same capacity, especially for the upload bandwidth above 25 KB/s. There is a slight variability for lower uploaders, because as high uploaders finish at different times for different experiments there is a
0
Figure 6.6: Uniform: rate convergence to 90% capacity for 40-50KB/s leechers
0
Figure 6.7: Uniform: rate convergence to 90% capacity for all leechers
different amount of seed bandwidth that becomes available for the remaining leechers.
Figures 6.12 to6.15 show that the other systems have much less correlation between a leecher’s upload capacity and its completion time, and much higher variability in download times for the leechers of the same upload capacity. Figure6.15shows many lower-contributing leechers obtaining faster download times than higher-contributing
10
Figure 6.9: Uniform: High-uploaders’ download time
leechers when using BitTyrant. Figure 6.14shows no correlation between completion times and upload capacities for the leechers with capacities of above 25 KB/s when using PropShare.
One might think that, because of a strong correlation between upload capacity and completion times, the better performance for high-contributing FairTorrent leechers
0
Figure 6.10: Uniform: Download time of all the leechers
0
Figure 6.11: Uniform: FairTorrent download time
comes at the expense of the low-contributing ones. This is not the case. Figure 6.10 shows the worst download times were 1347, 1892, 1849, 1758, and 2266 seconds for FairTorrent, BitTorrent, Azureus, PropShare, and BitTyrant, respectively, with Fair-Torrent 31% to 68% faster. There are several reasons that FairFair-Torrent performance for lower contributors does not suffer. First, FairTorrent guarantees a certain level
0
Figure 6.12: Uniform: BitTorrent download time
0 Figure 6.13: Uniform: Azureus download time
of performance to each leecher. Each leecher receives a matching leecher download rate plus an equal fraction of the seed bandwidth. Thus, a leecher does not get stuck because of unlucky initial peer selection, or poor seed allocation. As a result, the slow-downloading outliers that we observe for other systems, in Figures 6.12 to6.15, are much less likely to occur in FairTorrent.
0
Figure 6.14: Uniform: PropShare download time
0
Figure 6.15: Uniform: BitTyrant download time
The second reason for better overall performance is the higher bandwidth utiliza-tion in FairTorrent. Figure 6.16, which plots each leecher’s average upload rate vs.
each leecher’s upload capacity in FairTorrent, shows that FairTorrent leechers achieve close to perfect bandwidth utilization. In contrast, Figures6.17 to6.20 show that all the other networks have a number of leechers that lie far below the ideal reference line
and have poor utilization. Those leechers could not achieve the desired upload rates.
Overall, the aggregate bandwidth utilization across all leechers for each network was 95.3%, 93.7%, 89.8%, 88.7%, and 82.8% for FairTorrent, BitTorrent, Azureus, Prop-Share, and BitTyrant, respectively. Part of the under-utilization is the necessary BitTorrent protocol overhead which accounts for roughly 3% in each network.
FairTorrent achieves higher utilization for all the leechers, because the FairTorrent leechers experience very few data availability bottlenecks. There are two aspects of FairTorrent that contribute to fewer bottlenecks. First, the seeds in FairTorrent allocate their bandwidth evenly among all leechers, increasing the probability that each leecher gets some seed bandwidth and thus an influx of data blocks it can then upload to other leechers. Second, FairTorrent leechers do not use unchoking and rate-allocation constraints that contribute to bottlenecks. FairTorrent unchokes all of the neighbors interested in its data, and always uploads a data block as long as at least one peer requests it. Other clients place a number of constraints on data exchange among the peers: (1) First, other clients allow uploads only to several peers in each round. As a result, there is a higher probability that a client may run out of data blocks to send to its small neighborhood. (2) PropShare and BitTyrant clients allocate uneven rates to their neighbors, which are enforced by additional limits on the data that can be uploaded to each individual peer. (3) BitTyrant uses Block-based TFT with other BitTyrant peers, which often causes a high-uploading peer to stop uploading to its neighbors.
It is interesting to note, that the reason that the non-FairTorrent clients use various constraints, such as unchoking of several peers at a time, and uneven rate-allocation, is because these clients rely on these constraining policies as a way to discover better data-exchange partners. FairTorrent, on the other hand, converges very quickly to fair exchange rates with its neighbors, and avoids the problem of having to discover
0 10 20 30 40 50
0 10 20 30 40 50
Upload Rate (KB/s)
Upload Bandwidth (KB/s)
Figure 6.16: Uniform: FairTorrent utilization
0 10 20 30 40 50
0 10 20 30 40 50
Upload Rate (KB/s)
Upload Bandwidth (KB/s) Figure 6.17: Uniform: BitTorrent utilization
like-uploading peers. As a side-effect, FairTorrent is able to unchoke all neighbors, and reduce the data availability bottlenecks.
Figure 6.21 shows the standard deviation in download rate observed by each leecher, as measured over consecutive 15 second intervals. Data points represent-ing FairTorrent are captured by the lowest layer on the graph, clearly separated from
0 10 20 30 40 50
0 10 20 30 40 50
Upload Rate (KB/s)
Upload Bandwidth (KB/s) Figure 6.18: Uniform: Azureus utilization
0 10 20 30 40 50
0 10 20 30 40 50
Upload Rate (KB/s)
Upload Bandwidth (KB/s) Figure 6.19: Uniform: PropShare utilization
the rest. The average standard deviations in leecher download rates were 1.8 KB/s, 6.0 KB/s, 8.0 KB/s, 6.7 KB/s and 12.3 KB/s for FairTorrent, BitTorrent, Azureus, PropShare, and BitTyrant, respectively, with FairTorrent’s standard deviation more than 3 times smaller than the next closest system. Because of its fast and more precise rate convergence, FairTorrent is able to maintain a download rate close to its
0 10 20 30 40 50
0 10 20 30 40 50
Upload Rate (KB/s)
Upload Bandwidth (KB/s) Figure 6.20: Uniform: BitTyrant utilization
0 10 20 30 40 50
0 10 20 30 40 50
Standard Deviation
Download Rate (KB/s)
TY AZ PS BT FT
Figure 6.21: Uniform: Standard deviation of the download rate
upload contribution throughout the download. The low standard deviation makes FairTorrent more amenable for use in P2P live streaming systems, where achieving and maintaining a steady download rate of stream updates is critical for good per-formance.