Data centers slowly shifting to converged networking

Though uptake of FCoE might be slow, those adopting the protocol know what they will get from it.

Though uptake of FCoE might be slow, those adopting the protocol know what they will get from it.

The Fibre Channel over Ethernet (FCoE) standard is fully baked, but user adoption is taking hold slowly. Most IT managers are interested in the concept of moving Fibre Channel traffic over Ethernet networks, but few have so far flipped the switch and put FCoE to use.

The FC-BB-5 working group of the T11 Technical Committee unanimously approved a final standard for FCoE in June of 2009. As a result, the T11 Technical Committee plenary session has forwarded the FC-BB-5 standard to INCITS for further processing as an ANSI standard.

According to the Fibre Channel Industry Association (FCIA), the FCoE products in OEM qualification today are based on the completed standard and users will be able to benefit from standardized FCoE solutions from day one.

But that is the future. Fibre Channel over Ethernet is still in the very early stages of development.

InfoStor magazine, one of Cabling Installation &Maintenance’s sister publications, has been tracking FCoE deployment plans closely over the past several months. In a reader survey last April, approximately 9% of respondents say they planned to roll out FCoE in 2009, while 33% of those polled had FCoE deployments in plan for 2010 or 2011. However, the overwhelming majority (58%) says it has no plans to use the technology.

InfoStor posed the same question to readers in October and the results were similar. There has been a slight rise in the number of users planning FCoE this year (13%), but almost 57% are standing pat with no deployments planned.

However, there are a select few organizations ahead of the adoption curve and they are experiencing the benefits of converged networking and the resulting cabling reduction first hand.

FCoE and the next generation data center

One of the few IT professionals on the bleeding edge of FCoE adoption is Kemper Porter, systems manager in the Data Services Division of the Mississippi Department of Information Technology Services.

Designing and building an efficient data center is a top concern for Porter. His department is in the midst of planning a big move to a new data center and expects to be up-and-running in six months. One of the items on Porter’s agenda is simplification. Part of that plan includes a transition to converged networking.

“We simply want to clean up when we get to the new building. We know we are going to have a very high rate of growth. We had to set a new precedent and deployment pattern,” says Porter.

Porter and his team essentially function as a service provider to various state agencies, provisioning servers and IT resources to application developers across the state.

Porter envisions a massive proliferation of VMware virtual machines (VMs) deployed as building blocks that look and feel like mainframes, all of which connect to centralized storage via converged network adapters (CNAs) with centralized data backups and disaster recovery at the server level.

Converged network adapters consolidate the IP networking capabilities of an Ethernet network interface card (NIC) with the storage connectivity of a Fibre Channel host bus adapter (HBA) via FCoE onto a single 10 Gigabit Ethernet interface card.

Porter is currently using three single-port CNAs from QLogic in conjunction with dual Cisco Nexus 5000 Series FCoE-capable network switches.

The CNAs are being used in a test and development capacity – Porter is waiting until the move to the new data center in 2010 to put them in critical roles. He says the move to CNAs is one of necessity.

“When you have a 3U-high server it does not have an integrated switch and you wind up with a proliferation of network cards – six Ethernet connections per server and two for Fibre Channel,” he says. “Having all of these network cards creates a spider’s den of all these cables going in and out of the servers. It creates an excellent opportunity for physical mistakes and makes the process of troubleshooting more difficult.”

Despite his aversion to so-called bleeding edge technologies, Porter is confident that the CNAs with FCoE will meet his future needs.

“There isn’t anybody really using it, but I believe touching it and working with it is the only way to get your confidence up. I would not describe myself as an early adopter. This is just about as much fun as I can handle. If I were not moving to a new data center I probably would not be doing this,” Porter says.

The old way of doing things is a non-starter for Porter. His rack servers can comfortably house 77 VM instances physical server, while maintaining mainframe-like reliability, but none of it would be possible without minimizing network adapters and port counts.

Porter now runs two-to-three connections to each server. “It brings the complexity way down. We will still have our Fibre Channel infrastructure with one connection rather than two per server and we still have redundant pathways because we connect to two Nexus switches,” he says.

Converged networking is the way of the future, at least for Porter. “We will now buy CNAs to put in all of our future VMware servers. That is a fact. The technology is solid enough that we are going that way,” he says.

Consolidating cabling with FCoE

Salem Hospital is one of Oregon’s largest acute care hospitals and operates the busiest emergency department in the state. As part of a recent data center upgrade, the not-for-profit hospital beefed up its business continuity plans, reduced energy consumption and upgraded its IT infrastructure to accommodate data growth from electronic medical records (EMR) and picture archiving and communications system (PACS).

Heng Him, chief technologist, Salem Hospital, says all of those needs were met using server virtualization in conjunction with 10 Gigabit Ethernet connectivity and CNAs.

The hospital runs a myriad of IT systems, but its newest gear includes a pair of Cisco Nexus 5020 switches with FCoE support. The Nexus 5020 switches connect over lossless 10 Gigabit Ethernet to the hospital’s newer servers, which host multiple healthcare applications and Microsoft Exchange.

The Nexus 5020 separates the storage traffic from data traffic, sending the storage traffic to Cisco MDS-based storage area networks (SANs) and Ethernet traffic to the Cisco Nexus 7000 switch at the core. Another pair of Cisco Nexus 5020 switches connects to existing Gigabit Ethernet servers, by way of Cisco Nexus 2000 fabric extenders.

Through the use of the Cisco networking gear and FCoE, Salem Hospital reduced the number of cables needed for each server by half, from four to two. In addition, the Cisco Nexus 5020 switches are deployed close enough to the hospital’s servers to allow for the use of relatively inexpensive twinax cabling.

“We cut our cabling and associated costs in half. We are using twinax instead of standard SFP+ cabling, which saves on cost,” he says. “Simplifying the switching platform and wire management also makes troubleshooting much simpler. If you look at our old racks versus the new, it is like night and day. Some [people] look at the new setup and ask where all of the cables are.”

Him says the hospital is also consuming less energy. “It’s clean and easier to manage and there are fewer hot spots. We bring the heat guns in regularly to check. We are experiencing better airflow and cooling now.”

Lastly, the hospital’s staff is spending less time managing separate domains in the data center, freeing them up to work on new projects that directly relate to patient care, including a new fetal monitoring program, a patient alert system and an upcoming Citrix virtualization deployment.

The separate IT teams that historically managed storage and networking have been trained on the Cisco converged networking platform.

Him estimates that his IT department saves up to four hours per day on SAN management. “Our team members can now cross-train and collaborate more effectively,” says Him.

Like the Mississippi Department of Information Technology Services, Salem Hospital plans to virtualize everything going forward with VMs, CNAs and FCoE over 10 Gigabit Ethernet.

“With FCoE we have grown from 50 servers to 300 in the last 18 months. The speeds we have gained from using FCoE allow us to throw virtual servers at every project. We are upgrading almost all of our applications, taking them off of physical servers and putting them on virtual servers,” he says.

Kevin Komiega is a contributing editor for Cabling Installation & Maintenance and senior editor of InfoStor magazine, which covers storage networking.

More in Data Center