VMware SD-WAN Cloud Gateway Advantages

(1)

VMware SD-WAN Cloud Gateway Advantages

A Real-World Look at How Cloud Gateways Ensure

Application Performance for SaaS and IaaS

(2)

Simulated VoIP Test from Lab Agents to a ThousandEyes Cloud Agent 8 AWS S3 File Download 10 Link Aggregation with No WAN Degradations 10 Impact of Packet Loss on AWS File Download 12 Browser Page Load of PDF on SharePoint 14 Impact of WAN Degradations on Web Browser Performance 15 Summary . . . . 17

(3)

Introduction

VMware SD-WAN™ is offered as a hosted solution in which VMware provides and manages VMware SD-WAN Orchestrator and a globally distributed set of cloud-based VMware SD-WAN Gateways as a service. This is the most common enterprise deployment where customers leverage the cloud Gateways to optimize their SaaS traffic such as Microsoft 365 and connect to IaaS platforms such as AWS. The hosted solution provides all the SD-WAN benefits and leverages the network of cloud Gateways positioned globally to provide the branch VMware SD-WAN Edges an “on-ramp to the cloud.” The cloud Gateways have close proximity to SaaS applications. They enable branch Edges to use multiple WAN circuits to establish overlay tunnels to the cloud Gateways and forward SaaS traffic through them; the cloud Gateways will therefore provide link aggregation and Dynamic Multipath Optimization™ (DMPO) for SaaS traffic destined for the Internet.

DMPO provides optimizations among VMware SD-WAN Edges and VMware SD-WAN Gateways, providing continuous monitoring, dynamic application steering, on-demand remediation, and application-aware overlay QoS.

In this document, we will demonstrate the advantages of VMware SD-WAN Gateways in optimizing SaaS applications and real-time traffic through their abilities to protect a branch’s last mile connectivity and aggregate the capacity of WAN circuits. We observed the following while testing the respective applications with synthetic traffic:

• In a scenario in which SaaS traffic is served through the cloud VMware SD-WAN Gateways, although that may introduce 5-15 ms of latency depending on the Gateway location, we can see from the test results that it has little to no impact on SaaS applications such as Microsoft SharePoint or a file download from AWS S3 because VMware SD-WAN provides consistent performance.

• In the event there’s packet loss in the WAN network, even with a single link the VMware SD-WAN Edge can perform packet duplication and overcome the loss. This is shown by the stable and unaffected MOS score of synthetic RTP streams going through the VMware SD-WAN Edge with 7% loss on its single WAN link. It similarly overcame jitter as well.

• When the VMware SD-WAN Edge has more than one WAN link, it can aggregate the WAN throughput and use both links for web applications and other TCP-based traffic.

The total time for a client using a dual-link Edge to download a 25 MB file from Amazon S3 was more than 40% faster than the baseline.

Where Gateways are located

VMware SD-WAN incorporates a distributed network of cloud VMware SD-WAN Gateways deployed at top-tier cloud data centers around the world. They include both IaaS POPs and co-location facilities covering over 30 regions globally, with more than 2000 Gateways deployed. The Gateways are strategically placed to provide close proximity to major cloud services and protect cloud and SaaS applications from SD-WAN branches and data centers. Cloud Gateways are hosted in locations with concentration of SaaS and IaaS vendors and provides low latency handoff to these services. Therefore they increase reliability for users consuming those services from the SD-WAN fabric.

The cloud VMware SD-WAN Gateways are offered as a managed service. They serve as the tunnel end for VMware SD-WAN Edges to establish overlay tunnels for forwarding of cloud-bound user traffic. Although proximity to branches is secondary, the Gateway POPs are typically less than 25 ms from last-mile ISPs. They enable VMware SD-WAN Edges to aggregate capacity of multiple links and maximize the effect of DMPO by protecting traffic from a branch all the way to the doorstep of SaaS and IaaS vendors.

(4)

Advantages of cloud Gateways beyond performance

Besides the VMware SD-WAN Gateways’ ability to protect cloud applications from the SD-WAN fabric, the globally distributed pool of cloud Gateways delivers several other advantages in terms of architectural availability, ease of deployment, and a flexible yet scalable solution to deliver cloud services. Here are some of the capabilities and benefits the Gateways provide beyond performance:

• Delivers instant global footprint on day 1

• Horizontally scalable and assures high availability for control plane functions

• Provides optimizations to SaaS/IaaS applications on data-center class networks

• Leverages upstream peerings at the POPs and inherently builds in network redundancy and failover

• Pre-built and eliminates the need to build DIY endpoints close to business-critical SaaS/

IaaS services

• Managed as a service and therefore offers scheduled upgrades, data redundancy and disaster recovery, and rapid-response operation teams across multiple locations

• No need to configure access to individual SaaS applications and no re-architecture or planning required if a new cloud application comes online

• Optionally provides aggregated connectivity to IaaS and non-SD-WAN sites

How Edges are assigned cloud Gateways

Given that SaaS and other business-critical Internet traffic traverses the cloud VMware SD-WAN Gateways from the VMware SD-WAN branch, it is critical that the cloud Gateways are optimally assigned to the VMware SD-WAN Edges to minimize the latency from the branches’ Internet circuits to the cloud Gateways. When a VMware SD-WAN Edge is provisioned and activated, it goes through the following sequence to be assigned primary and secondary cloud Gateways for forwarding of cloud traffic:

• Upon provisioning of the VMware SD-WAN Edge, the geo-location of the branch can be configured on the VMware SD-WAN Orchestrator’s dashboard.

• If the address was not configured, the Orchestrator will attempt to geolocate the Edge based on the Edge’s public IP address.

• Base on the Edge’s geographical location, the Orchestrator assigns the closest cloud Gateway to the Edge as the primary Gateway and the second-closest cloud Gateway as the secondary Gateway. The Edge uses the primary Gateway to forward and optimize SaaS traffic. In addition, two other Gateways elected for the enterprise are assigned to the Edge for route propagation and other control plane functions.

• The Edge receives the cloud Gateway’s information from the Orchestrator and establishes overlay tunnels to the four cloud Gateways.

Testing topology and methodology

The lab topology is outlined in the diagram below where the four ThousandEyes

Enterprise Agents generate synthetic traffic to measure and compare the performance of user applications with SD-WAN and without SD-WAN. The Enterprise Agents are provisioned as virtual machines in the same lab infrastructure and therefore observe the same underlying network connectivity to the SaaS applications in question.

Agent #1 is considered a baseline. It will not experience any simulated network degradation during the testing so we can maintain a consistent view in case anomalies occur on the server side or outside of the lab environment. Agent #2 simulates a user with no SD-WAN. Its traffic will follow the underlay path to the tested SaaS applications and directly leverage the WAN connections on the firewall. Agent #3 sits behind a VMware SD-WAN Edge with a single WAN link, therefore its traffic to the cloud and SaaS

(5)

applications will be forwarded by the Edge through the SD-WAN fabric to the VMware SD-WAN Gateway. The single-link Edge will leverage VMware DMPO on its overlay to the Gateway to remediate against packet loss and jitter in the network path, and demonstrate how SD-WAN can benefit the user applications on Agent #3 even when there’s only one WAN link available. Both Agent #2 and Agent #3 are connected to the same WAN emulator so we can see how VMware SD-WAN remediates against the packet loss and jitter that the emulator generates. The last Agent in the testing is connected to a VMware SD-WAN Edge with two uplinks, demonstrating DMPO’s ability to aggregate bandwidth across multiple links and how it can steer latency-sensitive traffic away from link

degradations in real time. Each link used by the ThousandEyes Enterprise Agent or the Edge is allocated 20 Mbps of throughput up and down on the router upstream from them to keep the environment consistent.

FIGURE 1: Testing topology showing the four performance monitoring agents

ThousandEyes agents and the test types

ThousandEyes is a SaaS platform that offers a variety of vantage points for enterprise network monitoring and application performance measurements. Per ThousandEyes product documentation, they offer a variety of vantage points from which you can run layered monitoring tests to gain insight into network and application performance and user experience. The monitoring agents, which include the Cloud and Enterprise Agents, can be configured to run various tests with synthetic traffic periodically to a target. The various test types can be used to monitor DNS resolution, browser response

characteristics, detailed aspects of network routing and connectivity, and VoIP connection quality. Data collected over time by the ThousandEyes Agents are reported to a

Dashboard for visibility, monitoring, and alerting.

For the purpose of this white paper, four Enterprise Agents were deployed in a simulated network as shown in Figure 1 to represent vantage points from four different scenarios.

The same collection of tests was run from the Agents. Depending on the type of test, the Agents were configured to target SaaS applications or existing ThousandEyes Cloud Agents to collect network and application performance metrics. The Agents were configured to target the same resources while the WAN emulator introduced identical link

PA Single Link Edge

PA Dual Link Edge

(6)

degradations to compare the performance of the monitoring Agents with and without SD-WAN in the path. Network Test, Web Test, and VoIP Test metrics were the primary focus as we examine the results to understand how VMware SD-WAN benefits user experience in real-world scenarios.

Tests and testing methodology

Given the environment shown in the topology above, the monitoring agents generated synthetic traffic targeting the tests, which are configured every 5 minutes, and recorded the application performance and metrics over time. After we collected enough samples over a 24-hour period, 7% packet loss of impairment and 10ms of jitter in both directions were added to the links on the WAN emulators with exception of GE4 on the dual-link VMware SD-WAN Edge. Another 24 hours of performance metrics were gathered after the packet loss was injected to measure the application performance of Agents #2, #3, and #4 with impairment. Packet loss and jitter were configured for the testing because they are the most prevalent form of degradation we experience. Other factors such as delay and packet re-ordering were not used, to simplify the results and analysis. The Baseline Agent was kept the same to detect any anomalies in the results.

Three synthetic tests were run on the Agents where we measured performance of applications with no WAN degradation and then added 7% packet loss bi-directionally:

• Figure 2 shows synthetic RTP streams from the four lab Agents to a ThousandEyes Cloud Agent on the Internet using G.722 audio codec. The goal is to simulate VoIP calls from a corporate WAN network to an external VoIP endpoint. Each source Agent sends a predefined number of UDP packets with RTP streams encapsulated and measures the Mean Opinion Score (MOS), which is achieved by calculating loss, discards, latency, and jitter. The voice test was configured with a duration of 10 seconds and de-jitter buffer size of 40 ms.

FIGURE 2: MOS scores for synthetic RTP streams from the monitoring agents

• Figure 3 shows HTTPS-based download of file from the same AWS S3 bucket and measures the time to completion to download 25 MB of the large file. To keep the S3 bucket destination consistent, the target was statically mapped to the S3 bucket in the us-west-2 region. The test is configured with a timeout of 60s. The Agent will measure the initial HTTP response time as well as the total time it took to download the 25 MB of data. The goal of the test is to simulate the experience of a user downloading a large file from a web server.

(7)

FIGURE 3: Test results for HTTP-based file download from the monitoring agents

FIGURE 4: Output from monitoring agent loading a SharePoint PDF file in its web browser

• Figure 4 shows page load of a PDF file from SharePoint in a web browser to measure the user experience of a client opening the document. The goal is to measure the real-life user experience of Microsoft 365 and how the document is presented to the user along with other web components on the page. The browser the Agent uses is Chromium based. Both DOM load and page load time will be in focus in terms of metrics. A sample of how the page is loaded on the Agent in a waterfall view from the HAR file is shown in the diagram below.

(8)

Test results

Here we will analyze and examine the test results and demonstrate how the cloud VMware SD-WAN Gateways benefit SaaS applications and improve user experience when

degradations occur in the WAN. As mentioned earlier, the results will show data collected over 48 hours where the synthetic tests were generated every five minutes. The data collected from all four Monitoring Agents were collected simultaneously and 7% packet loss were injected by the WAN emulators at 00:00 UTC on May 28th; additionally, 10ms of jitter in both directions was injected by the WAN emulator at 06:00 UTC, May 28th. There were 24 hours of data collected under normal conditions and another 24 hours of data collected after simulated WAN degradation was in place. We will examine in detail results from each test below.

Simulated VoIP Test from Lab Agents to a ThousandEyes Cloud Agent

The goal of the test is to measure the MOS score of the synthetic RTP streams the agents generate, and compare the scores before and after packet loss and jitter are introduced.

First, we will look at the MOS score when no WAN degradation is in place as shown in Figure 5. Note the table is showing detail results from the time slice between May 27, 23:00-23:05 UTC, which is before WAN degradation was injected at 00:00 UTC. The MOS score for the simulated VoIP test from all four Agents in question measured 4.3 as G.722 is a high-quality audio codec consuming about 64 kbps of bandwidth.

FIGURE 5: MOS score from monitoring agents before introduction of packet loss

With 7% packet loss introduced by the WAN emulator at 00:00 UTC, by switching over to the loss metric view in a timeline and examining the data collected at 00:15, we can see in Figure 6 that only the No-SDWAN Agent was observing the 7% loss while the Agent behind the single-link VMware SD-WAN Edge was seeing minuscule impact. This is because the single-link Edge utilized packet duplication to overcome the loss on its WAN overlays to the cloud VMware SD-WAN Gateways and remediated the packet loss. Agent

#4 behind the dual-link Edge was not impacted at all as it was able to steer around the loss detected on GE3 and steer the RTP stream on a per-packet basis to its GE4 link which was not experiencing degradation.

(9)

FIGURE 6: Comparison of packet loss observed by SD-WAN agents vs Non-SDWAN agent when packet loss is introduced

Looking closely at the MOS scores recorded as shown in Figure 7, we can see the No-SDWAN agent’s MOS score dropped to a range of 4.0-4.05 after it started observing the 7% packet loss. Agent #3 behind the single-link VMware SD-WAN Edge was minimally impacted as the MOS score dropped slightly to 4.29 in this sample of data. VMware’s DMPO kicked in and packet duplication for real-time traffic was applied. The MOS score from Agent #4 behind the dual-link Edge was not impacted as the Edge simply avoided using the SD-WAN overlays established on the impaired WAN link GE3. Comparing the MOS score over time between the No-SDWAN Agent (Agent #2) and the Agent behind the single-link Edge (Agent #3), we can see the drastic improvement on Agent #3. The No-SDWAN Agent was not even able to complete the test during multiple intervals due to the loss, in addition to the drop in MOS score. The reason MOS score from the

No-SDWAN agent is not worse is because the target Cloud Agent where the synthetic traffic is received is configured with a jitter buffer of 40ms, and the bandwidth requirement of G.722 at 64 kbps is relatively minimal compare to the WAN link throughput of 20 Mbps.

This test and the results demonstrated that VMware SD-WAN provides benefits to real- time applications such as VoIP by applying DMPO techniques to remediate loss, jitter, and other WAN degradations seen across the overlay tunnels.

(10)

FIGURE 7: MOS score results for monitoring agent behind single-link SD-WAN versus no SD-WAN after packet loss

AWS S3 file download

In this test, the lab Agents are configured to download a file from Amazon S3 and measure the amount of time it takes to retrieve 25MB of the file via HTTP protocol. The destination S3 bucket was statically mapped to us-west-2 and the VMware SD-WAN Gateway assigned to the VMware SD-WAN Edges in question is located in the San Francisco area.

There are several interesting facets to this test that we will dive deeper into. First, we often get questions from customers regarding extra latency introduced by the VMware

SD-WAN Gateways, given that the traffic traversing the VMware SD-WAN Edge to the cloud will have to be forwarded to the Gateway through the SD-WAN overlay – which adds an extra hop. With the results we will observe the actual impact on user experience from the small amount of latency. Secondly, given the traffic type is TCP and the 25 MB of data is downloaded via HTTP, we can see the effect of link aggregation used by DMPO when an Edge has multiple WAN links.

Link aggregation with no WAN degradations

Starting with the results when no WAN degradations are in place, we can see the results from the right most column in Figure 8 below. The third column shows the total time it takes to download the 25MB of data and the measured throughput in Mbps. Under normal conditions and given all the agents are limited to 20 Mbps, No-SDWAN Agent #2 and the single-link SD-WAN Agent #3 observed the same performance as the Baseline Agent as it took about 13.3 seconds for the Agents to retrieve the 25 MB. However, we can see Agent

#4 behind the dual-link Edge was able to aggregate the throughput on its two WAN links, and the total time to download the file was about 6.3 seconds – more than 50% faster than the Baseline agent. The results demonstrated the ability for DMPO to utilize multiple WAN links and perform link aggregation for TCP-based traffic flows. This functionality comes with the solution out of the box and applies to applications that benefits from aggregation, such as file transfer.

(11)

FIGURE 8: Demonstration of link aggregation when performing file download from Amazon S3

FIGURE 9: Latency and jitter measurements from the monitoring agents to Amazon region us-west-2

Given that Agent #3 behind the single-link VMware SD-WAN Edge performed just as well as the baseline and Non-SD-WAN agents, it’s interesting to look at the latency data from the diagram below and observe that latency from Agent #3 to the web server is actually 5-10 ms higher than the baseline on average when no WAN impairment is injected. This can be seen from the timeline data shown in Figure 9 below where Agent #3 behind the single-link Edge is highlighted in a blue line versus the average latency for all the probes.

Notice that even though latency for the Agents behind the Edges are slightly higher, per the table in Figure 8 above, the overall HTTP response time measured by TCP Connect Time, SSL Time, and Wait Time was not affected and the total download time for Agent #3 was about the same as the baseline. This data demonstrated that even though cloud Gateways may introduce slight latency, VMware SD-WAN overlay tunnels provide consistent performance and do not negatively impact user experience in terms of application performance.

(12)

Impact of packet loss on AWS file download

Surveying the results after the 7% packet loss was introduced, we see some obvious benefits of how VMware SD-WAN is able to improve the file download experience. Using data seen during the time slice at 01:45 UTC shown in Figure 10 below, we can see the No-SDWAN Agent observing the packet loss was unable to finish the transfer and it was only able to download about 8 MB out of the 25 MB of data within the 60 second window.

This confirms the negative effect of packet loss on TCP windowing and how severely it can affect web-based data transfer.

FIGURE 10: Throughput and download time of a S3 file transfer from the monitoring agents

In fact, if we look into the data points over time during this 48 hour window for Agent #2 with no SDWAN, per Figure 11 below we can see it was never able to fully download the 25 MB of data after the loss was injected, and each periodic test ended with HTTP receive error – note the error message “Operation timed out after 60…”. The timeout for the test was configured at 60 seconds. We can see that Agent #2 was consistently unable to complete the 25 MB download within the given interval.

FIGURE 11: Agent with no SDWAN unable to complete file transfer after packet loss introduced

(13)

For Agent #3 behind the single-link VMware SD-WAN Edge, we can see that even though the agent was able to complete the download, per Figure 10 the total time went from 13.3s to almost 40s. The lengthy transaction time was likely a result of throughput drop due to packet re-transmissions and DMPO remediations. In this specific use case, DMPO contributed to the improvement over No-SDWAN Agent by applying NACK to TCP based traffic flows. That dramatically improved the performance even though only a single WAN link was available for the file transfer. By examining the packet loss observed on the end client (Agent #3) in Figure 12 below, we can see that even though 7% symmetrical loss was introduced on the WAN upstream from Agent #3, the monitoring Agent did not observe the packet loss from the end client perspective because the single-link Edge in the path was able to remediate the degradation by employing various DMPO techniques, including NACK.

FIGURE 12: End-to-end packet loss observed by monitoring agents to and from Amazon S3

For Agent #4 performing this synthetic file transfer behind the dual-link Edge, the data indicates it was observing similar results as the Baseline, because the dual-link Edge upstream from the agent simply avoided using the depredated WAN link and steered the traffic to the WAN overlay established on the better link on GE4. The dual-link Edge does not perform aggregation because with severely degraded overlay paths such as the tunnels established on GE3 link with the packet loss, the Edge will mark the paths as unusable and steer traffic away from the link until underlying conditions improve.

Figure 13 below demonstrates this behavior. We can see that after packet loss was introduced on one of the WAN links around 00:00 UTC, Agent #4 was seeing having the same download time as the baseline agent observing no degradations. Before the packet loss was introduced, the dual-link Edge allowed the monitoring agent to double its throughput with link aggregation.

(14)

FIGURE 13: Amazon S3 File transfer performance from monitoring agent behind VMware SD-WAN Edge with dual WAN links

In summary, the median performance metrics for the monitoring agents after packet loss injection are shown in the table in Figure 14. They clearly illustrate the performance enhancements VMware SD-WAN provides to TCP-based applications such as a file transfer as it performs real-time steering of traffic and remediation techniques to alleviate WAN degradations.

FIGURE 14: Performance metrics for the S3 file transfer after packet loss injection

Browser page load of PDF on SharePoint

In this this test we will validate the benefits VMware SD-WAN brings to Microsoft 365 and analyze how WAN degradations impact user web experience from a real-world

perspective. Using a Chromium-based web browser on the monitoring agents, the test was configured for the lab Agents to load a multi-page PDF file in the web browser so we can compare the HAR file results. From the results we will look at the actual browser performance metrics such as DOM load time and Page Load time to determine the differences and improvements with the Agents connected to the SD-WAN fabric.

Keeping in mind that 7% packet loss was injected on the WAN emulator at 00:00 UTC on May 28, we will first look at the browser DOM load and page load time of the PDF file before the degradation as shown in the diagram below. We can see the metrics were about the same for all the Agents and the Page load time was about 1000 ms for a total of 70 objects on the page. As the packet loss was injected a few minutes later, we can observe the increase in average page load time in the timeline data shortly after the time slice.

(15)

FIGURE 15: Browser performance and page load time of a SharePoint file from monitoring agents

Impact of WAN degradations on web browser performance

By selecting the No-SDWAN Agent #2 and studying the packet loss data and the page load time collected by the Agent after packet loss was injected, we can observe the dramatic increase in page load time from the browser’s perspective and how packet loss may influence an end user’s impression of a web application. Figure 16 and Figure 17 show that once 7% packet loss was injected on the WAN link, page load time from the

no-SDWAN Agent increased substantially while neither Agent #3 nor #4 behind the SD-WAN fabric were affected. It is very interesting to see how detrimental packet loss can be in terms of a user’s web browsing experience, which can be comprised of various factors such as DNS, SSL, HTTP, and backend performance.

FIGURE 16: Observance of packet loss from the monitoring agents during testing

(16)

FIGURE 17: Impact of packet loss on web browser page load from monitoring agent with no SDWAN

Digging into the HAR file as shown below, we can see how the packet loss caused load time of a few components on the page to take a hit. In return, that critically impacted the page load time. This was a fairly consistent behavior after the packet loss was injected and later on when jitter was introduced.

Comparably, if we look at the median page load time of Agent #3 behind the single-link VMware SD-WAN Edge, we can see the browser performance on the SDWAN-single-link Agent is significantly better than the metrics without SD-WAN – a median of only 1.4s versus 7s for Agent #2. This is due to VMware SD-WAN applying DMPO in real time and improving the TCP transaction with NACK and other techniques within the SD-WAN overlay. Lastly, as expected, WAN degradation on the dual-link Edge’s WAN circuit did not affect the user experience of the monitoring Agent behind it, and it performed just as well as the baseline over hundreds of data points. The dual-link Edge was able to effectively steer traffic away from the impacted link without causing user impact.

FIGURE 18: HAR file from monitoring agent with no-SDWAN and observing packet loss

FIGURE 19: Median browser performance metrics for SharePoint file’s page load test after packet loss injection

(17)

Summary

In conclusion, this document demonstrated the advantages of VMware SD-WAN Gateways in optimizing SaaS applications and real-time traffic through their abilities to protect a branch’s last mile connectivity and aggregate the capacity of WAN circuits. We provided various proof points to illustrate how the Gateways enhance connectivity to the cloud and improve real-world user experience. The cloud VMware SD-WAN Gateways offer consistent performance with on-ramp to the cloud, and enable SD-WAN branches to apply DMPO to remediate against WAN degradations for both real-time and TCP-based applications.

VMware SD-WAN mitigates WAN challenges today with unique application-aware DMPO technologies and a globally distributed set of cloud Gateways to ensure consistent performance and an outstanding user experience for business-critical applications such as Microsoft 365 and public cloud services. The unique ability of DMPO to protect a branch’s last mile connectivity and the solution’s advanced cloud infrastructure enables

organizations to gain simple to deploy, secure, and high-performing WAN connectivity from branches to the cloud.

(18)

VMware SD-WAN Cloud Gateway Advantages