There are several possible algorithms that provide congestion control that is TCP fair in that its long term bandwidth utilization is equal to TCP or that its utilization when under large amounts of loss must not exceed TCP. Such algorithms could be rate or window based. This value could use a formula or a reactive system or a combination of the two to determine the limit. Clark and Tennenhouse discuses the design considerations for different network protocols in [32].
6.2.1
Window tracking systems
A window tracking system reacts to events using a function to change the congestion window in response to these events.
There are a class of congestion control algorithms, called Binomial Congestion Control Algorithms[9], that are window tracking and increase and decrease their window based on these equations. This set of congestion control algorithms includes TCP.
I:wt+R←wt+α/wtk;α >0
D:wt+δt←wt−βwlt; 0< β <1 (6.2) WhereI is the increase function andD is the decrease function,wt is the window value at timet,R is the RTT andα,β,kandlare constants. A binomial algorithm is TCP-compatible if and only ifk+l= 1 andl≤1. Fork= 0 and l = 1 the resulting system is AIMD, like TCP in the congestion avoidance state. Fork =−1 and l = 1 we get Multiplicative Increase Multiplicative Decrease (MIMD) which is what TCP uses in the slow start phase. Chiu and Jain discuss different increase and decrease functions in [27].
The congestion window provided by these algorithms can then be used in one of two ways. It can be used directly to limit the number of packets in flight at anyone time or it can be converted into a rate.
Window based Apart from TCP there are other possible window tracking based controlled congestion control systems.
A system could use the same or similar increase and decrease functions but be packet rather than byte based. For example, Datagram Congestion Control Protocol (DCCP)’s Congestion Control ID (CCID) 2 [57] is designed to approximate the behaviour of the congestion control system used by TCP [160, 4]. A congestion window is maintained to control the number of packets that are in the network at anyone time, as DCCP is packet-oriented rather than byte-oriented, like TCP, the congestion window is in units of packets rather than bytes. TCP does not explicitly provide congestion control of the acknowledgements, though a change in the rate of data packets will also change the number of ACKs. CCID 2 provides for the ratio of ACKs to data packets to be changed when the loss of ACKs is detected. CCID 2 requires that acknowledgements contain an ackvector option that indicates the arrival status of packets.
Figure 6.4: Different loss rates.
The window tracking congestion control system provides congestion control that is similar to that provided by TCP. It reacts quickly to changes in the network conditions and is TCP fair. It uses the most bandwidth possible whilst staying TCP fair. It is good for traffic that has large amounts of data to move but does not need a steady transfer rate.
Rate based These systems would work by using the window value maintained to calculate a rate at which to send packets. This can be achieved by limiting the number of packets or bytes that can be sent in a period of time or by waiting a calculated amount of time between sending packets.
6.2.2
Formula controlled systems
The accuracy of determining the rate or probability of an event dependence on the probability of the event. High probability events can be more accurately and more quickly determined. Using old events means that when the probability changes from high loss to low loss this is detected slowly as old loss events are kept for a long period of time. This results in:
• Loss can be determined more accurately when it is high so algorithms more responsive.
• Loss is difficult to determine when low so algorithms not responsive.
• Loss is over estimated unless it is very high.
An equation based congestion control system based on loss must calculate the probability of loss so that it can determine the correct window size. To prevent oscillation an average is used. In TCP Friendly Rate Control (TFRC) this is a weighted average of 8 congestion events. A loss event starts with a packet loss and then continues for up to one RTT of lost packet. If a packet is lost less than one RTT from the beginning of the loss event it is part of that loss event. The congestion events are recorded as the number of packets from the beginning of the loss event to the beginning of the next loss event, or the current packet for the current unfinished congestion event. This includes the loss event, the lossy part, and the lossless part following it. The congestion events are recorded as a lossy part and a lossless part. The weights for 8 loss events in TRFC are 1.0, 1.0, 1.0, 1.0, 0.8, 0.6, 0.4, 0.2. The values are multiplied by the weights and added together and divided by the total of the weights, this gives the mean distance between losses. The value is calculated for 8 events including the current and for 8 excluding the current one, the larger mean loss interval is used.
Without any averaging it would take a number of packets equal to the loss event rate over 1, the loss event length, to determine the loss rate. With an averaging system using 8 loss events it takes 8 times the number of packets for the value to exactly match the actual value. This means it takes 4 times as many packets to determine that the loss event rate has halved than it does to determine that the rate has doubled. This is illustrated in figure 6.4. The white squares are successfully received packets and the red are lost packets.
If the loss rate changes from one value to another the length of time that this takes depends on the sending rate, so the packet loss before the change. When the change is from a low loss rate to a high loss rate the faster packet transmission rate will cause it to approach the correct value more quickly. The calculated sending rate for a connection when the loss rate has changed from 5% to 0.1% is shown in figure 6.5a. The calculated sending rate when the loss rate has changed from 0.1% to 5% is shown is figure 6.5b. These figures show the difference between a window tracking and equation based congestion control systems in regard to speed of convergence to the correct sending rate. The number of congestion events remembered by the system affects the rate at which the system converges to the correct sending rate. A smaller history will lead to it reaching the correct level more quickly, however a smaller history will also lead to the sending rate being more sensitive to a change in the loss rate.
0 20 40 60 80 100 120 140 160 180 0 200 400 600 800 1000 1200 1400 packet/RTT Time (RTTs) Window tracking 1% Formaula based 1% Window tracking 0.1% Formaula based 0.1% Window tracking 0.01% Formaula based 0.01% (a) From 5% 0 20 40 60 80 100 120 140 0 10 20 30 40 50 60 70 packet/RTT Time (RTTs) CWND 0.01% rate TFRC 0.01% CWND 0.1% rate TFRC 0.1% CWND 1% rate TFRC 1% (b) To 5%
Figure 6.5: Calculating the sending rate after a change in loss rate.
Flight size The number of packets in-flight is important as well as the number of packets transmitted per unit time. The congestion window limits the number of in-flight packets so that the number of packets in the network do not overflow the buffers on the routers in the network. The amount of packets on the network divided by the amount of time it takes for those packets to reach there destination and be acknowledged, the RTT, determines the rate at which the packets are delivered and so also the rate at which they are sent onto the network. The AIMD window tracking congestion control of TCP and others converges on a fair share of the number of in flight packets.
The TCP fair equation determines the rate at which a connection will, on average, be sending packets when it has reached a steady state with packets in flight in the network. If a connection is able to transmit packets onto the network a steady rate for sufficient time, without network conditions changing, it will achieve its calculated transmission rate. If the amount of congestion on the network decreases the RTT will decrease and the in flight packets will be delivered faster. Connections with more packets in-flight than would be necessary to achieve their sending rate without congestion will then be able to more quickly adapt to congestion.
Rate Based An example of a rate based formula systems is DCCP’s CCID 3 [58], which is a slight variation of TFRC [72]. It is designed to compete fairly with TCP whilst providing a more smooth packet sending rate. It uses the following formula to determine the correct sending rate:
X = s
R∗p
2∗b∗p/3 + (t RT O∗(3∗p
3∗b∗p/8∗p∗(1 + 32∗p2))) (6.3)
X is the rate at which data is to be transmitted at in bits/second.
sis the packet size in bytes.
Ris the round trip time in seconds.
pis the loss event rate, between 0 and 1.0, of the number of loss events as a fraction of the number of packets transmitted.
t RT O is the retransmission timeout value in seconds that TCP would have calculated.
bis the number of packets acknowledged by a single TCP acknowledgement.
The packet transmission rate is also limited to doubling once for each feedback packet received.
Window based The formula is used to calculate the congestion window value to be used by the congestion control system. It would still require that packets were acknowledged as they arrived otherwise it would not be able to keep the correct number of packets in the network. Such a system would react to changes in the RTT without having to recalculate the value of the congestion window.
6.2.3
Design Choices for traffic management system
In the existing Second Life traffic control system the rate limiter (RL) on the server side of the connection limits the rate at which data is sent from the server to the client. The Second Life rate calculator (SLRC) on the client end of the connection calculates a rate and sends it to the server. The structure of the system is shown in figure 6.6.
Figure 6.6: The existing system.
The new system involves changing the client but not the server, so it will change the value sent from the client to the server. This limits the type of congestion control system that can be used to those that are rate based rather than window based. There are two types of rate based congestion control that were discussed in sections 6.2.1 and 6.2.2. Equation based congestion control systems adapt to changes in the loss rate more slowly than window tracking systems. Equation based systems will reach a stable level when conditions on the network do not change. Window tracking systems reach a stable equilibrium between an upper and lower bound. An equation based system would then work well for a system that prefers stable rate over the highest mean transfer rate.
The Second Life rate system is in bits per second which means that the congestion control systems will have to use bits and not packets as its unit. All of the types of congestion control system need a packet size in order to give values in bits and not packets. TCP uses the maximum segment size, however TCP is designed so that it will not send out large numbers of packets that are not the maximum segment size. The number of packets as well as the number of bytes is important so a packet size that accurately reflects the size of the packets being sent is important. The average packet size would therefore seem like the correct value to use, however the mix of packet sizes is not constant so a pre-calculated average may not produce correct results.
Second Life circuits also have sequence numbers which can be used to calculate the number of packets lost. The loss detection system used by Second Life is a timeout based system that operates at the receiving end of the connection and relies on detecting gaps in the sequence range. The timeout is 16/1000 times the average ping time in milliseconds, which is 16 times the round trip time. The system keeps track of the sequence number that it expects to see next. If this sequence number is seen then the value is incremented otherwise the sequence number is searched for in the list of sequence numbers of potently lost packets. If it is found this sequence number is removed from the list. If it is not found, and the packet is not marked as having been re-sent, all of the sequence numbers between this one and the expected one are added to the list of possibly lost, and the next expected sequence number is set to this packet sequence number plus one.
The new system, implemented at the client, will be a different rate calculation system called TCP compatible rate calculator (TCRC). TCRC is a rate based window tracking system, maintaining a congestion window and then calculating a sending rate from this window value and RTT. The loss detection system in Second Life does not detect two different type of loss event, just the one. The window tracking system used is modelled after TCP, it maintains a congestion window (CWND) and a slow start threshold (SSTHRESH). The implementation therefore only uses one type of loss event. When loss is detected the CWND and SSTHRESH are reduced to half the CWND. When a packet is received that is not out of sequence the CWND is increased. When the CWND is less than SSTHRESH the CWND is increased by one average packet size. When the CWND is larger than SSTHRESH then CWND is increased by the packet size squared divided by the CWND. This is the commonly used CWND growth function used by TCP [4]. The CWND is supposed to grow by one segment per RTT, this formula approximates that behaviour. It would have been possible to increase window by one segment for each window of packets by counting the number of packets since the CWND was last increased. It is also possible to record the time when CWND was last increased and increase it by one segment every RTT. The method used is a commonly used method and is the simplest method as it requires the least state. The rate is calculated when the system is idle, the rate is calculated by dividing the CWND by the RTT. The rate at which this information is sent to the server will also be important. The interval at which this calculation is performed is also important. The value can be recalculated every time a packet is received or every time that a packet is received or every time that packet loss is detected. The system could alternatively calculate the sending rate at a set interval or when the loss rate or RTT changes by a large enough value. The system used for this calculation
Figure 6.7: The data-flow in the client and server.
will be the window-rate system discussed in the previous section. Such a system would have the advantage of only requiring that the client was changed. The normal design for a congestion control system is to have the system that calculates the rate at which packets or bytes are to be sent so that it can react without the need for the other end of the connection to tell it to.
The periodic calculation method will provide a stable transfer rate whilst adapting to network conditions. The existing system will be replaced with a window tracking system to calculate a throttle total value, this value will then be filtered to prevent sending updates for small changes in the network conditions. The calculated sending rate will be put into the existing function, that calculates the individual channel throttles from overall throttle value. The new system will provide a sending rate that is TCP fair.