Our data center network simulator consists of the C++ source, NED and message files, and a set of support utilities to create a light-weight OMNeT++ simulation model for FatTree DCNs. Here we provide a brief overview of the simulator and its components and provide a high-level view of how we modeled FatTree DCNs.
Dependencies
Our project uses OMNeT++ version 4.3 which is free for non-commercial use and runs on Windows, OS X, and most Linux/Unix systems.
Some of the modules in our code also depend on the following:
• INET 2.0.0 [4]. Note that to create an OMNeT++ project using our code, INET must be included in the project’s workspace and listed as a project reference.
• Boost C++ libraries [1]. We used boost version 1.52 and we link our code against the boost system and boost filesystem libraries.
Directory structure
The root directory consists of three folders:
• src: the root folder for all of the simulation modeling files (i.e., NED files and their corresponding C++ code)
• BuildFatTree: contains a simple command line application to generate NED files for FatTree topologies.
• simulations: contains ini files for several example experiments as well as some python scripts for results processing.
The structure of the src folder containing the model files is as follows:
• common: includes commonly used headers and also houses the base classes that are common to multiple parts of the project.
• networks: this folder includes the NED files defining the data center topologies.
• node: all of the server code is kept here and its subfolders correspond to the layers of the networking stack.
• packets: contains the OMNeT++ message definitions and C++ source that define all the packets and control messages used.
• switch: contains the components that make up the network switches.
A.2.1 Modeling FatTree DCNs
Here we briefly describe our representations for the various elements that make up our FatTree DCN simulation model.
Packets
Since we did not use the INET models for protocols like Ethernet and IP, we model these protocols by creating simple message types that capture their essential behavior.
We created the following message definitions to represent the different packet types for each protocol:
• DCN EthPacket: We extended OMNeT++’s cPacket type to model Ethernet packets. We customized the automatically generated C++ class produced by the message definition so that we could account for the minimum payload and the various header fields in the Ethernet frame when setting the packet size.
We assumed the standard MTU of 1500 bytes to determine the maximum size of Ethernet frames.
• DCN IPPacket: We extended the DCN EthPacket type to create a message type for IP packets. It adds 20 bytes to account for the size of the IP header fields. In addition, we assumed that we could overload existing IP options fields in order to account for the sequence numbers and timestamps used by our resequencing approach. As a result, we added an additional 8 bytes to the IP overhead.
• DCN UDPPacket: To represent UDP datagrams, we extended the DCN IPPacket type. It simply extends the packet size accounting logic to include the overhead of the UDP header.
We did not create a separate TCP packet type since we rely on INET’s TCP model to support TCP which already defines its own TCPSegment type. To support this message type in our framework, we use the cMessage class’s encapsulate/decapsulate feature to transport TCPSegments inside DCN IPPackets.
Links
To model network links, we created a DCLink class which is an extension of the
standard OMNeT++ cDatarateChannel class. This class automatically simulates transmission delay by using the data rate of the link and the packet’s size field. The links can support any bit rate but we primarily used a value of 1 Gbps for all of the links in the network. The links can also simulate propagation delay as well as channel noise by introducing random bit errors. However, we did not use these features in any of our simulations.
Switches
We assume switches are store and forward switches (i.e., no cut-through switching).
We model the switches as ideal output-buffered switches by placing a queue in front of each port and by directing packets into the queues corresponding to their destinations as soon as they arrive. The routing logic at switches is hardcoded specifically to route packets across the FatTree topology. While there are a variety of different approaches to construct FatTree DCNs from existing switches, we do not tie our model to any particular approach and only assume that some mechanism exists to allow servers to choose paths for their packets. We model this by including a “path” field in the DCN EthPacket class that can be set by a server enabling it to indicate to the switches which path the packet should take. Dropping at switches occurs when the output queue grows beyond the defined limit, which can be set in terms of bytes, packets, or both.
Servers
Since this thesis focused on software-level mechanisms, the majority of the complexity of our simulation model resides in the server nodes. We structured the server nodes according to the layers of an operating system’s networking stack. We devote section A.3 to describing each of the key modules that make up our server model.
Networks
In the network folder, we provide a set of NED files (SubFatTree.ned, FatTree.ned, and ServerNode.ned) to recursively construct FatTree networks by generating a compound module to represent each subtree. However, we found that this approach was not very efficient because each time a packet traverses the network, it must be placed on the event queue every time it passes between the compound modules corresponding to different subtrees. We found that it was far more efficient to define one network compound module that directly contains all of the switches and servers as submodules.
As a result, we wrote a stand alone application called BuildFatTree to automatically produce NED files for different FatTree networks. While the link speed is configurable, the program only produces fully provisioned FatTree networks (i.e., full bisection bandwidth) that are constructed from links of the same speed. The program also supports the ability to produce the logically equivalent tree topology for a given FatTree network, as described in Chapter 3.
A.2.2 BuildFatTree
BuildFatTree application:
The BuildFatTree application is written in C++ and can be built by entering the BuildFatTree folder and typing “make”. Running the application without any argu-ments will cause its usage to be displayed. Once a NED file is produced, it should be placed in the src/networks folder so that OMNeT++ can find it. Note that the OMNeT IDE attempts to index all C++ and NED source files which can cause it to slow down dramatically when processing large files. For this reason, the generated NED files for large networks should not be placed in the working directory or the IDE should be instructed not to look in the path containing these files.