This subsection describes the testing procedures we perform, and the methodology and reasoning behind them. We also outline the goals of our testing procedures.
To recap, these are the design goals we aim to achieve with our request rerouting mecha- nism:
1. Able to work around DNS caching/prefetching 2. Able to fully utilise network information 3. Easy to configure
4. Incrementally implementable
5. Able to utilise and act on the ability to rapidly create virtual CDN replica servers 6. Scalable for network and service expansion
7. Able to provide access to the CDN for end users that are not customers item of the ISP
From these, the ones we can directly test are goals 1, 2, and 5. The testing procedures we thus create must therefore be able to not only test for goals 1, 2, and 5, but also ideally test for the system performance in achieving those goals 10. To that end, the following test procedures were created:
• Request rerouting after link failure • Request rerouting during link congestion
• Request rerouting with the introduction of a new CDN replica server
Also, within each of these procedures, we define two types of tests: Passive user and Active user. Because the ultimate goal of a CDN is to increase QoE and QoS for their end users, it is therefore desirable to model ‘expected’ user behaviour in times of duress. As such, we model a scenario where the user tries to access content via simply typing the URL into their browser, and another scenario where if congestion is experienced, or load time is greater than 20 seconds, the user refreshes the page.
3.3.2.1 Request rerouting after link failure
In this test, we observe the effectiveness of our proposed request rerouting mechanism after a CDN replica server experiences a link failure. Here, we are primarily concerned with content access delay.
First, the end user visits thesis.com, which points to CDN replica server h2. Doing so causes them to perform a DNS request for h2. They are assumed to stay on the landing
page of thesis.com for 10 seconds while they read some of the news. During this time period, a shell script is called that disables the link between h2 and s1, and updates the controller via the REST API that h2 has gone down, and that requests for h2 should be redirected towards h3. At the same time, the DNS server at h5 updates it’s A record to reflect that fact. After the end user has stayed on the landing page of thesis.com for 10 seconds, they then click on a hyperlink to thesis.com/firstpage.html, which contains more content. We then time how long it takes for that page to successfully load. In the passive user case, if the page takes longer than one minute to load, we end the test. In the active user case, the user attempts to refresh the page if there are no signs of activity after 20 seconds.
3.3.2.2 Request rerouting during link congestion
In this test, we observe the effectiveness of our proposed request rerouting mechanism when our CDN replica server is under heavy congestion.
In this scenario, we simulate the link between s1 and h2 (The switch and a CDN replica server) as being congested. This is achieved by lowering the bandwidth on the link between s1 and h2 to 1Mbps. As with the previous test, thesis.com initially points to h2. The test begins when a user visits thesis.com. The user stays on thesis.com for 10 seconds, and is then redirected to thesis.com/firstpage.html. During this 10 second period, a shell script is executed that notifies the controller(c0) that CDN replica h3 is available, and uncongested. Correspondingly, the DNS server at h5 updates it’s A record for thesis.com to point to CDN replica h3. We then time how long it takes for the user to complete a full trip around the thesis.com testpages. After a full trip, when the user is back on the thesis.com landing page, we then time how much longer it takes to for the user to connect to the new CDN replica server(h3) if they have not already done so.
3.3.2.3 Request rerouting with the introduction of a new CDN replica server This test can be seen as an extension of the previous test. The purpose of this test is to examine the impact that the proposed request rerouting mechanism might have on users who are not located in the ISP’s network, but yet are still using the CDN service 11. This
test will also provide insight on how rapid establishment of a new CDN server depending on server demand can have an effect on both the ISP’s customers, but also visitors to the ISP’s CDN network.
In this scenario, both h1 and h4 (an end user in the ISP network, and an end user not in the ISP network) are initially connected to h2. We make the link between h2 and s1 constrained to 1Mbps, so as to easier simulate the effects of congestion. As with previous tests, both hosts visit the landing page of thesis.com initially, and then wait for 10 seconds. They then visit thesis.com/firstpage.html, causing the link between s1 and h2 to become congested. At the same time that this link becomes congested, a 30 second timer is started. At the end of this 30 second timer, h3 is ‘brought online’, and the controller is notified that there is now a new, more optimal CDN replica server available. Throughout these tests, we are timing how long it takes for h1 and h4 to reach
the initial thesis.com landing page again, compared to the case where we don’t use this thesis’ proposed method of request rerouting.
Chapter 4
Results & Performance
This chapter contains the results and performance of our proposed request rerouting mechanism. Observations relating to the testing process are also made.
4.1
Request rerouting after link failure
The results of 50 tests are shown in Table 4.1 below:
Firefox Chrome mean std mean std Reference mean load time 22.8 0.05 22.7 0.08 With improved request rerouting mechanism 27.2 2.2 26.8 2.4 Without improved request rerouting mechanism; active user 75.3 or 115.9 1.52 or 0.6 144.3 4.196 Without improved request rerouting mechanism; passive
user
N/A N/A N/A N/A
Table 4.1: Time taken to fully traverse and load the thesis.com testpages, in seconds.
The reference time is the time taken to load the website directly from the CDN replica server, without any form of redirection.
With both Firefox and Chrome, when the browsers were left alone as described in passive testing mode, both failed to load a page after over a minute, and so are both counted as timeouts.
Interesting behaviour was observed for the active testing mode in both Firefox and chrome. In Firefox, two different, but locally similar sets of results were gathered; One with a mean total load time of 75.3 seconds, and another with a mean total load time of 115.9 seconds, a 40 second difference. This was caused by two different behaviours during active testing:
• A refresh was performed after 20 seconds of inactivity, as outlined in Subsec- tion 3.3.2.1. Upon receiving no result, another refresh was done at 25 seconds, 30 seconds, etc. until the website finally started loading. Activity was consistently observed at 100 seconds after thesis.com was initially loaded. This behaviour cor- responds to the 115.9 mean load time.
• A refresh was performed after 50 seconds of inactivity (60 seconds total since the start of the testing procedure). Activity was consistently observed immediately after this reload. This behaviour corresponds to the 75.3 mean load time.
As shown in the table, the average mean load time for Chrome is at least 30 seconds higher than that for firefox. The active behaviour was similar - refresh at 20 seconds, then 25, 30, etc. until activity is observed. In this case, the page access delay was a minute; when the page was refreshed after a total of a minute’s delay since the browser visited thesis.com, the first testpage appeared to start loading. However, the page didn’t fully load - Only seven images out of the ten would load on firstpage.html. After 20 seconds, the page was refreshed and the page loaded fine; but the same behaviour occured on secondpage.html, as well. This behaviour appeared consistently, and contributed majorly to the extra time observed in Table 4.1.
The results from this testing procedure indicate that our proposed request rerouting system can successfully reroute end users after a link failure. However, there is an ob- servable performance cost, with mean load time taking around 5 seconds more with our proposed mechanism compared to the reference. However, after the 10 second delay on thesis.com, the access delay until firstpage.html started loading was negligible.