It might boil down to one word: reach. But along the way there’s a lot to consider.
Several considerations that go into whether to deploy a multimode-fiber-based, short-wavelength optical network or a singlemode-fiber-based, long-wavelength optical network in a data center environment were discussed in an online seminar that took place on April 5. Hosted by Cabling Installation & Maintenance, the seminar was titled “Optical Fiber Advances and Capabilities.” It will be available for viewing on-demand until October 5, 2018.
Presenters Tony Irujo, sales engineer for optical fiber with OFS, and David J. Asta, senior data center applications engineer for Panduit, delivered information about multimode fiber options and singlemode fiber options, respectively. Each presentation covered cabling technologies as well as standards activities and other efforts by industry consortia to develop solutions for high-speed optical networking in data centers.
Reach and other factors
The decision to use multimode/short-wave or singlemode/long-wave infrastructure could be boiled down to one word: reach. If your optical links need to be only a certain distance, multimode is the way to go; once that distance is exceeded, it’s singlemode. However, it’s a bit more nuanced than that. And on the way to finding that reach, data center network planners must take into consideration a number of other words and phrases, including multiplexing, parallel, multisource agreements, leaf-and-spine, insertion loss, and oh-by-the-way, economics. The remainder of this article will draw primarily from the information Irujo and Asta delivered on April 5, as well as from information produced by others.
“For many years multimode fiber has served the industry well as a cost-effective solution for short reach,” Irujo stated. “Today OM4 is a workhorse multimode fiber, capable of 10-Gigabit reach to about 600 meters and 40/100-Gigabit transmission in the 300-meter range, depending on the transceivers used.”
Irujo explained that OM3 and OM4 fiber “support transmission using a single wavelength, primarily at 850 nanometers. To achieve higher data rates, you can add fibers; these additional fibers allow you to aggregate to higher speeds” through parallel optics. For example, one iteration of 40-Gbit/sec Ethernet calls for the use of eight multimode fibers, in which four fibers each transmit 10 Gbits/sec, and four others each receive 10 Gbits/sec. The concept can “ratchet up” to 100-Gbits, with each of four fibers transmitting 25 Gbits/sec and the other four each receiving 25G.
“Parallel or multifiber transmission is a good way to increase data rates, but just to a point,” Irujo noted. “When you start talking about 32 fibers to support 400G, it gets unwieldy.”
An alternative to higher-fiber-count multimode constructions is the use of OM5 fiber and transceivers that facilitate wavelength-division multiplexing. “OM5 allows you to transmit multiple wavelengths on one fiber,” Irujo explained. He added that wavelength-division multiplexing (WDM) is “a technology that has been used with singlemode fiber for many years. It’s called SWDM—short-wavelength division multiplexing. OM5 is designed for performance not just at 850, but across the range from 850 to 953 nm,” he said.
“SWDM on OM5 fiber can provide duplex 100G links and allow for 400G operation using the same eight-fiber technology that’s currently in use for 40 and 100G links,” according to Irujo.
Standards and specifications
On the standards front, OM5 is defined in two standards—TIA-492AAAE and IEC 60793-2-10 ed. 6. It also is referenced in ANSI/TIA-568-3.D and ISO/IEC 11801-1. As for applications standards that call for the use of OM5, it’s a mixed bag. Irujo pointed out that both IEEE activity as well as an industry multisource agreement (MSA) can advance the deployment of SWDM.
“A next-generation multimode fiber study group within the IEEE has been looking at more practical options for 400G data rates,” he said. “One option involves four pairs of multimode fibers in a parallel configuration, with each fiber carrying two wavelengths and each wavelength transmitting 50G. Another option involves eight pairs of multimode fibers, with each fiber using one wavelength, carrying 50G. It’s worth noting that this is the first time the IEEE has looked at multiple wavelengths on multimode fiber,” he emphasized. “This sets the stage for future WDM solutions using multimode fiber.”
Furthermore, the SWDM Alliance was established in 2015, and the SWDM Multi-Source Agreement (SWDM MSA) Group published two specifications in March 2017. One spec defines SWDM-based 40-Gbit/sec signals and the other defines SWDM-based 100-Gbit/sec signals.
During the April 5 seminar, Panduit’s Asta explained that an MSA can be characterized as “a consortium of companies that put together multiple-source agreements [MSAs] to design and manufacture—in these cases—transceivers. While they are not part of the IEEE, they follow the specifications for Ethernet in the construction and design of these transceivers. They are, however, separate entities—not part of the IEEE.”
Historically, some but not all of the work originated via MSAs has been adopted in IEEE specifications.
While OFS’s Irujo pointed to SWDM-based 400G activity within the IEEE, a recent blog post by Leviton’s senior director of product management, Gary Bernstein, reported on a 200G SWDM-based effort that did not advance within the IEEE. “At the March 7, 2018 IEEE 802.3 meeting, the Next-Gen Multimode 200- and 400-Gbit/sec study group voted down a physical layer specification that supports 200-Gbit/sec operation over one pair of multimode fiber. This was the second time the proposal was voted down. The specification would have included duplex OM3, OM4, and the potential for OM5, which supports SWDM.”
The post points out that a hurdle facing OM5/SWDM-based applications is, “SWDM technology cannot be easily broken out at the servers, limiting it to switch-to-switch topology. Yet most installed multimode fiber links in data centers—almost 50 percent according to Leviton data—use breakout cables at switch to server. This severely limits the broad market potential for 200 Gbits/sec over duplex multimode.”
Nonetheless, Irujo emphasized in his April 5 presentation that cost considerations favor SWDM and OM5 in many cases, and particularly in short-reach applications. “The cost of optics—the transceivers—dominates the link,” he said, adding that depending on the application, singlemode optics can cost anywhere from 2 to 5 to 12 times multimode optics, when comparing average street prices.
The singlemode solution
Panduit’s Asta pivoted the conversation toward the characteristics and benefits of singlemode-based, long-wavelength networking in data centers. The conversation begins with a look at the landscape of hyperscale facilities. “Not long ago, if someone was talking about a ‘large’ data center, you’d probably be talking about a total of 250,000 square feet. Today we’ve seen data centers grow in size, to a 1.5-million-square-foot Google data center, to [a facility located in] Langfang, China, which is 6.3 million square feet.
“How are we going to support data centers that large? Singlemode cabling helps us address these needs,” Asta continued. “There’s a lot of leaf/spine deployment. The concept is that every leaf is connected to every spine, and that consumes a lot of fiber.” In a hyperscale facility—even one that doesn’t quite incorporate 6.3 million square feet—reach is a consideration when a leaf/spine configuration is deployed. “The spine and leaf may be several hundred meters apart,” Asta explained. “It’s not a matter of a leaf residing in one cabinet and a spine right next to it. These are applications where longer-reach solutions will be advantageous.”
Asta pointed out that for many large facilities the proverbial sweet spot for reach is longer than the distance where short-wave/multimode systems max out, and shorter than the 10-plus-kilometer reach of applications like 100GBase-LR4 and -ER4. IEEE standardization efforts as well as MSAs are addressing this need. Specifically, Asta pointed to projects specifying 500 meters and 2 kilometers. They include IEEE 802.3cd 50GBase-FR (2 kilometers), 802.3ba 100GBase-DR (500 meters), 802.3bs 200GBase-DR4 (500 meters), 802.3bs 200GBase-SR4 (2 kilometers), 802.3bs 400GBase-DR4 (500 meters), and 802.3bs 400GBase-FR8 (2 kilometers) from the IEEE. Additionally, several MSA consortia have made efforts toward specifying 100G singlemode, including the PSM4, CWDM/CWDM4, and CLR4 groups.
Singlemode-based long-wavelength transceivers that achieve reduced reach can be manufactured more economically than traditional 10-plus-km optics can, thereby lowering costs per link for the data center operator. The tradeoff for this lower cost is a tighter insertion-loss requirement for the installed cabling plant—in the range of 3 to 3.5 dB. “That’s do-able,” Asta advised, “but we have to be much more cognizant of the connectivity that we’re using in these applications. Go with connectivity that will be optimized or ultra-low-loss. It will pay dividends in the long run.”
One application that fits this description is 100G PSM4—100-Gigabit parallel singlemode using four lanes. Panduit is a member of the PSM4 MSA that published its specification in 2014. “An advantage is that it lends itself to a four-to-one breakout,” Asta explained.
Other MSA-based 100G singlemode solutions for the data center include a coarse wavelength-division multiplexing (CWDM) option that “allows you to maintain duplex singlemode without going to MPO [connectivity], and achieve 500-meter reach,” Asta noted. This technology has been adopted by the Facebook-led Open Compute Project. The 100G CLR4 specification enables a 2-km reach of 100G over duplex singlemode fiber.
Pinning down exactly when and where singlemode/long-wave is a better option than multimode/short-wave is not necessarily straightforward. One of the final points Asta made during the seminar was that in a conventional four-cassette 100G link scenario, once a link reaches a specific distance it becomes more economical to use singlemode/long-wavelength optics than to use multimode/short-wavelength optics. That crossover point is at or near 650 meters. Yet several 100G specifications produced via MSAs and the IEEE focus on 500-meter reach. The emerging technology eSWDM4 extends the reach of 100G over OM5 to 400 meters.
“Distance is a factor,” Irujo said during the April 5 seminar’s question-and-answer period. “For multimode, we’re looking at the few-hundred-meter range where economics come into play.”
Patrick McLaughlin is our chief editor.