Whitepaper
Introduction
In tests conducted at the Chelsio facility, results demonstrate successful interoperability between Chelsio’s latest S310E 10G adapter with twin-ax cabling and Arista Networks’ 7124S switch. See Figure 1. Performance results show that the application can sustain line rate and latency numbers are ~7 us.
Arastra’s switch along with Chelsio’s adapter provides a high throughput, low latency 10 Gigabit Ethernet based solution. The Arastra switch delivers an outstanding balance of performance and value with key data center class features. The latency and throughput performance shown in this report demonstrate that 10 Gigabit Ethernet is well suited for operation in demanding clustering applications.
Arastra is setting the standard for high throughput, low latency 10 Gigabit Ethernet switches. By providing the features data center managers require along with a vision of unified networks Arastra and Chelsio will help drive 10 Gigabit Ethernet adoption into the mainstream market.
System Configuration
Figure 1. Network Topology
System #1 configuration System #2 configuration
2 Intel dual-core Xeon 5160 @ 3.0 GHz 2 Intel dual-core Xeon 3070 @ 3.0 GHz
Memory 8Gb Memory 8Gb
OFED 1.3 OFED 1.3
RHEL 5.1 Kernel 2.6.18-53.eI5 RHEL 5.1 Kernel 2.6.18-53.eI5
Chelsio S310E Arista 7124S 10 GbE Twin-ax Copper Cables System #1 System #2
Performance Graphs
IMB (Intel MPI (Message Passing Interface) Benchmark) Sendrecv, Exchange Latency and Throughput
0.1 1 10 100 1000 10000 1 4 16 64 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 1 0 4 8 5 7 6 4 1 9 4 3 0 4 Bytes L a te n c y u s 0 500 1000 1500 2000 2500 T h ro u g h p u t M B /s
Sendrecv Latency Avg Exchange Latency Avg Sendrecv Throughput Exchange Throughput
IMB (Intel MPI (Message Passing Interface) Benchmark) PingPong, PingPing Latency and Throughput
0.1 1 10 100 1000 10000 1 4 1 6 6 4 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 1 0 4 8 5 7 6 4 1 9 4 3 0 4 Bytes L a te n c y u s 0 250 500 750 1000 1250 T h ro u g h p u t M B /s PingPong Latency PingPing Latency PingPong Throughput PingPing Throughput
IMB (Intel MPI (Message Passing Interface) Benchmark) Allgather, Allgatherv, Alltoall, Bcast Latency Avg
0.1 1 10 100 1000 10000 1 4 16 64 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 1 0 4 8 5 7 6 4 1 9 4 3 0 4 Bytes L a te n c y u
s Allgather Latency Avg
Allgatherv Latency Avg Alltoall Latency Avg Bcast Latency Avg
IMB (Intel MPI (Message Passing Interface) Benchmark) Allreduce, Reduce, Reduce_scatter Latency Avg
0.1 1 10 100 1000 10000 100000 4 1 6 6 4 2 5 6 1 0 2 4 4 0 9 6 1 6 3 8 4 6 5 5 3 6 2 6 2 1 4 4 1 0 4 8 5 7 6 4 1 9 4 3 0 4 Bytes L a te n c y u s
Allreduce Latency Avg Reduce Latency Avg Reduce_scatter Latency Avg
The Chelsio Solution
Chelsio’s All-In-One adapters are equipped with Chelsio’s flagship third-generation Terminator ASIC (T3). Due to the cut-through processing of the T3’s data-flow architecture the latency is kept low. Chelsio’s 10G solution:
• Lowers latency
• Lowers standard deviation of latency • Significantly lowers TCO
• Leverages existing network fabric infrastructure
• Scales over multiple nodes without any performance hit Summary
The results collected using the Intel MPI tool highlight that Arista’s 10G Ethernet switch is going to be a very strong player in the HPC space and as a very viable alternative to InfiniBand. Chelsio’s hardware is specifically designed to dramatically improve cluster performance by reducing the application latency while keeping the CPU utilization at a minimum.
About Chelsio Communications Inc.: Chelsio Communications is leading the convergence of networking, storage and clustering interconnects with its robust, high-performance and proven unified wire technology. Featuring a highly scalable and programmable architecture, Chelsio is shipping 10-Gigabit Ethernet and multi-port 10-Gigabit Ethernet adapter cards, delivering the low latency and superior throughput required for high-performance computing applications. For more information, visit the company online at http://www.chelsio.com
About Arista Networks: Arista Networks, based in Menlo Park, California, was founded to deliver scalable networking interconnects for large-scale datacenters and cloud computing. Arista offers
best-of-breed 10 Gigabit Ethernet solutions for Cloud NetworkingTM that redefine scalability, robustness, and
price–performance. At the core of Arista's platform is the Extensible Operating System (EOSTM),
pioneering new software architecture with self-healing and live in-service software upgrade capabilities. Arista's team is comprised of seasoned management and engineering talent bringing over a thousand man-years of expertise from leading network and server companies. For more information, please visit:
http://www.aristanetworks.com
Arista Networks, the Arista logo, Cloud Vision, Cloud Networking and EOS are trademarks of Arista Networks, Inc. in the U.S. and other countries.
The trademarks, logos and service marks ("Marks") displayed in this document are the property of Arista Networks, Inc. or other third parties. Other names and brands may be claimed as the property of others.
