IBM benchmark tests show 10G Ethernet outperforming InfiniBand

Results showed that 10-Gigabit Ethernet is superior in performance to InfiniBand in some standard performance benchmark test suites and comparable in others.

Sept. 22, 2010

2 min read

Add Us On Google

SUNNYVALE, Calif. -- According to a benchmark study recently conducted by IBM Corporation, 10-Gigabit Ethernet delivered better performance than InfiniBand and Gigabit Ethernet for a range of high performance computing (HPC) applications.

"We are pleased that IBM has published the results of its extensive performance testing for key HPC applications and the results clearly show that 10Gb Ethernet is superior to InfiniBand for a broad base of HPC applications and very similar in others -- validating the reasons behind the rapidly growing success of Ethernet as the HPC interconnect technology," said Kianoosh Naghshineh, president and CEO of Chelsio Communications, Inc.

According to a press release issued by Chelsio, IBM researchers compared the performance of several applications in HPC using 4x InfiniBand DDR, 10 Gigabit Ethernet using iWARP, and Gigabit Ethernet using TCP/IP. The results showed 10Gb Ethernet is superior in performance to InfiniBand in some standard performance benchmark test suites and comparable in others.

The test configuration was a computing cluster using a 2.3 GHz quad-core Opteron processor, with each node having 16Gb memory and two quad-core processors. Each node was configured with a 4x DDR InfiniBand connected to a 96-port Cisco DDR switch, using dual-port DDR ConnectX adapters from Mellanox; a 10Gb Ethernet network using Chelsio dual-port adapters with full offload capability, connected to a 20-port Force10 10Gb Ethernet switch; and a Gigabit Ethernet network connected through a Cisco switch. The software configuration was standards-based as well, running RedHat Enterprise Linux Server release 5.2.

IBM ran eight applications over the networks, including NETPERF -- a benchmark suite that is commonly used to measure various aspects of networking performance, with a primary focus on bulk data transfer and request/response performance using either TCP/IP or UDP and the Berkeley Socket Interface (BSI). The NETPERF results showed 10Gb Ethernet greatly outperforming InfiniBand for both TCP and UDP, and the Gigabit Ethernet was a distant third.

Full report: IBM Benchmarks 10Gb Ethernet and InfiniBand for HPC Applications (PDF)