The Institute of Electrical and Electronics Engineers' final approval of the 802.3an 10GBase-T standard was heralded as a bellwether event in the summer of 2006. The 802.3ae standard for fiber-based 10-Gigabit Ethernet (10 GbE) networking had been finalized four years earlier, and 10GBase-T's final approval meant users desiring optical or twisted-pair copper-based 10 GbE would have both options available.
Three years later, potential users of 10GBase-T now have the opportunity to deploy the protocol end-to-end. Early 2007 saw the announcement of the first 10GBase-T network interface cards (NICs), and these devices are now readily available from multiple sources. More recently, 10GBase-T switches have come onto the market, permitting twisted-pair copper-based 10-GbE connectivity between the server and switch.
In March, Extreme Networks introduced the Summit X650 10GBase-T switch. In May, Arista Networks announced its 7100T 10GBase-T switch line. In making its announcement, Arista emphasized two points: The product line's density and autonegotiation capability.
Arista offers either 24 or 48 ports of 10-GbE connectivity in a 1U switch. "Our 48-port models are the only 48-port 10-Gig products of any kind in 1RU today," says Mark Foss, Arista's director of marketing. "Many competitive products have 24 ports in 1RU. Nobody else does 48 ports of 10-Gig in 1U." Extreme's Summit X650 is available as a 24-port 1U switch.
Benefits of autonegotiation
"Also, many of our competitors' products are 10-Gig only; they are not autonegotiable," Foss continues, again referencing both copper- and fiber-based 10-GbE product lines. "We support both 1- and 10-Gig in every port."
A port's ability to autonegotiate between 1 and 10 GbE may influence the rate at which the protocol is adopted by user organizations. Autonegotiating NICs in particular may help spur the process.
Bob Kenny, General Cable Corp.'s vice president and general manager of data-communications, recalls what kind of impact autonegotiating 100/1000Base-T ports made on user adoption:
"When people were using 100Base-T, they began buying Gigabit cards. Slowly, users began to turn Gigabit on. The beauty of autonegotiation is that when users are ready for a higher-speed technology, it can be there for them."
The cost delta between Fast and Gigabit Ethernet ports that had shrunken since Gigabit Ethernet's introduction was an important factor that allowed 100Base-T users to deploy autonegotiating Fast/Gigabit Ethernet ports at a time when they only required the lower speed.
Cost and architecture
Without a 10GBase-T switch product currently available from market ruler Cisco Systems, there is no cost-per-port measurement against which other providers can compare themselves. Even so, Arista is positioning itself as a cost-effective option for 10GBase-T networking.
"We realize price is a factor," says Arista's Foss. "It's not everything, but it is one consideration for a user moving from one technology to another." He characterizes the 7100T's premium as "a little more than 2x" over Gigabit-speed port costs for top-of-rack, server-class connectivity.
The top-of-rack architecture has gained popularity over the past half-dozen years or so, Foss explains. Previously, the most-popular architecture placed one or more switches at the end of a row of racks; those switches connected all the servers in the row. With increasing frequency over that time, users have moved toward a top-of-rack architecture, in which multiple servers reside within a single rack with one or more switches at the top of that rack aggregating them.
Of significance to the cabling trade, the top-of-rack architecture makes for short server-to-switch connections. Arista makes the point that these short-reach connections, as well as any others that are 55 meters or shorter, can successfully run over Category 6 cabling.
The products in the 7100T line "are ideal for server connections since they enable a seamless transition from 1 to 10 Gigabit Ethernet without requiring new cabling," says Andy Bechtolsheim, Arista's chief development officer, when the company announced the line. As has been well documented in this publication, users planning to run 10GBase-T over Category 6 cabling should do so only after employing the alien-crosstalk mitigation techniques described in the Telecommunications Industry Association's Telecommunications Systems Bulletin TSB-155 Guidelines for the Assessment of Category 6 Cabling in Support of 10-Gigabit Applications.
Power hurdles
Power consumption is another hurdle that 10GBase-T will have to clear before it sees wide adoption. "At the highest level, 10GBase-T adds 4.5 to 5 watts per port to the power of a switch, versus SFP+ versions" of 10-GbE, says Arista's Foss.
General Cable's Kenny predicts, "in 18 to 24 months, significant advancements will be made in the amount of power that 10GBase-T devices consume."
George Zimmerman, chief technical officer with silicon vendor Solarflare Communications, backs up Kenny's view. Solarflare provides physical layer interfaces (PHYs) as well as server controllers for 10-Gigabit Ethernet systems. "We're sitting at a really interesting time," Zimmerman says. "The power consumption of what is being bought off the shelf today is at the high end of feasibility. But if you look at the OEMs' [original equipment manufacturers] designs and what might be termed 'in the oven,' you will see a very different story."
Zimmerman adds, "Today, monolithic devices, at full performance, use less than 6 watts, and typically in the 5-watts-and-under area," per port. That is in contrast to the performance of previous-generation multi-chip module devices that were used in 10GBase-T PHYs. Today's devices incorporate power-saving measures, so generally speaking, they consume a maximum of 5 watts per port. "There's also a dynamic scale that automatically saves power, down to about 4 watts, if less than 100 meters of cable is used," says Zimmerman. "So, the maximum consumption on a short line is about 4 watts. At 4 watts per port, a 24-port switch line-card, which is an industry-class density, produces about 100 watts of power from the PHY. … Those are devices that today are in active stages of product design and test in major OEMs."
Because Solarflare provides technology for servers as well as switches, the company and Zimmerman have insight into power consumption in that end of the network as well. Specifically, Ethernet controllers must similarly reduce power consumption, Zimmerman explains. Peripheral Component Interconnect Express (PCIe) cards "can use up to 25 watts," Zimmerman says. "OEMs say they really want them around 18 watts. With some upgrades in the controller area, we expect to be looking at 10 watts for these add-in cards next year."
Once the total per-port NIC consumption is as low as 10 watts, users pretty much "stop worrying about counting individual watts on the NIC," he comments. "As you get under 5 watts for PHYs, you stop counting the PHY silicon consumption in the switch."
Energy efficiency
In the past, Zimmerman has opined that autonegotiation is also an important consideration with respect to energy consumption. In his blog entitled Up and Down the Network Stack he has written the following.
In prior generations of Ethernet, [autonegotiating PHYs] proved to be the path toward rapid provisioning of new speeds. With multi-speed 10GBase-T NICs in their servers, an end-user may begin to provision 10GBase-T-capable servers prior to upgrading the switching infrastructure and later install the 10GBase-T switch and upgrade the entire set of servers' speeds. This allows for much easier and more-rapid provisioning without a forklift upgrade, where entire infrastructures are changed at once.
A second reason, growing in importance, for multi-speed PHYs is power management… I am talking about 'wake-on LAN' capability. Putting it simply, today's version of server power management puts idle servers into a low-speed mode (100Base-TX usually) and waits for a 'magic packet' to 'wake on LAN.' This way idle machines can dramatically lower their power consumption.
With the increasing EPA and industry focus on making devices consume minimal power when not doing work, this feature is a must for servers. Eventually we'll have to see how this plays out with regard to Energy Efficient Ethernet, but that won't occur [immediately]. Today if you want energy efficiency when a server is idle, you support wake-on LAN functionality.
Simple autonegotiation enables some important benefits which have helped the dominance of Base-T copper Ethernet. However, it is not to be forgotten that this is because it allows new generations to build on the prior generations' technological and installed infrastructure base of Base-T networks. Now that 10GBase-T is offering the lower speeds of Ethernet, we will begin to see the power of autonegotiation as it allows IT managers to asynchronously upgrade the speeds of their servers and still support power management, through wake-on LAN, for idle systems.
A conversation with Solarflare's Zimmerman is a journey deep into the engineering of devices that make networks perform and improve. With improvements expected in the areas of cost per port and power consumption, 10GBase-T is charting a path toward more widespread adoption.
FOLLOW OUR EDITOR ON TWITTER: twitter.com/cablingmag