A University of New Hampshire system independently monitors a cabling product's compatibility in a multi-vendor environment.
The University of New Hampshire's Interoperability Lab (www.iol.unh.edu) in Durham has developed test suites and services to help manufacturers verify the conformance and interoperability of Gigabit Ethernet and Fast Ethernet products. These test suites and services are tools by which each "consortium" develops and refines its testing environment. A consortium is an independent focused interest group.
The Interoperability Lab, or IOL as it is commonly called, primarily focuses on the first two stages of the International Organization for Standardization (ISO-www.iso.ch) Open Systems Interconnection (OSI) reference model. By addressing the physical layer, the IOL is able to validate that such tasks as the conversion of data into electronic signals, bit timing, and framing are all conformant to a set standard. Also, by testing devices in the data link layer, the lab can test to see that packets are delivered in order, without errors, and conform to a set standard.
The first stages of a conformance test suite are derived from the standards that govern a specific technology. Gigabit Ethernet and Fast Ethernet standards are read and a series of tests are written. These conformance tests are used to monitor and gauge a product's ability to perform various operations within the framework of a set standard. It is important for a product to conform to a set standard, because without it, a product might have trouble interoperating in a network or even struggle performing basic operations.
Interoperability test suites monitor the ability of a product to co-exist in a multiple-vendor environment and operate with other products. A company needs to make sure that its products are interoperable and competitive with alternative solutions. The Gigabit Ethernet and Fast Ethernet test suites let members verify that their products are interoperable and consistent with other vendors' products.
Evaluating standards
"We will be contacted by vendors, usually daily or weekly, asking us to begin looking at a given technology and offer a testing service," says Michael Froning, marketing manager at the Interoperability Lab.
"Typically, the first thing we will look at is the standard itself," explains Froning. "How is that standards body regulated? Is it a common standards body that we're used to working with, or are there no standards bodies involved?" The IOL will look at the technology and try to make a judgment call on how open that standards body is. Most of the standards with which the lab works are "open standards" so that it can foster a multi-vendor environment.
The IOL has two modes of testing. The first is called "classic interoperability testing." In this first mode, you have your device under test and you are looking at testing that is per a given reference environment. As Froning explains: "Companies 'B' through 'X' will be used to verify that company 'A' works with 'B' through 'X', using a common application that you would expect to see in the product's environment." After this is done, the lab will create a report detailing the results of the test. "This sounds pretty basic, but we offer quite a service because you are in a neutral setting where vendors 'B' through 'X' can keep and house their products," says Froning. "You won't have to spend a lot of money on testing your product."
Individual results for each vendor are kept confidential between the lab and the vendor. Also, during the week, a vendor can come in and use services at the lab to make sure that its products conform to the other vendors' products. Testing can be done on performance- or production-level products, or even a development or beta product. Members sign up for time slots as a security measure so that a competitor will not be using the lab at the same time. The IOL may offer an arena for vendors to communicate in an unbiased way to solve common problems. This use is determined by the vendor's needs, and any shared information is only given at the discretion of the vendor.
The second mode of testing takes more time. During this portion, the lab evaluates how the standard breaks down. "After we locate the 'bugs' in the product, we dig deeper and look at the conformance-level aspects," continues Froning. "We look at the standard, reading it word-by-word, and develop a test procedure for every 'must', 'shall', and 'may' statement." In addition to developing the test procedures, the IOL will introduce negative testing and "bogus" frame types. Many times, a standard will specify how a product should react to a negative test. Froning says to do that, "you have to have command of the layer below on the OSI model."
UNH's IOL hopes to use these modes of testing to fill in the gaps that standards bodies leave behind. "A lot of times, a standards body will create a technology that is so ambiguous and open-ended to accommodate all of the vendors' interests that they aren't defining a good level of functionality," explains Froning. The IOL will work together with standards bodies or with the vendors to solve interoperability problems. It provides feedback by telling where the companies may misconstrue a statement, or letting the standards body know the statement itself is too ambiguous. Test suites create guidelines detailing how a product should perform.
Physical and data link layers
Froning says that you must have "command of the layer below on the OSI model" to conduct negative testing-one of the ways in which the IOL evaluates a product or technology. Layer 1 is the physical layer and describes the physical properties of the various communications media, as well as the electrical properties and interpretation of the exchanged signals. For example, this layer defines the size of an Ethernet coaxial cable, the type of connector used, and the termination method. Layer 2 is known as the data link layer. It describes the logical organization of data bits transmitted on a particular medium.
Having the layers put together on the OSI model is known as a "stack." "When you do interoperability testing, you are actually testing the whole stack," says Froning. "Conformance-level testing focuses on Layer 1 or 2." Most of the technologies that the IOL test are Layer 1 and 2 technologies.
Gerard Nadeau, IOL's Fast Ethernet Consortium and Gigabit Ethernet Consortium manager, says "Interop 'test beds' are composed of equipment such as Ethernet cards and PCs from different vendors. We hook up the equipment over a near-worst-case cable plant channel and use a shorter connection. Then we will pass a certain amount of traffic so that we can determine a bit-error rate based upon a packet-error rate. The number of packets that are dropped will help us get a bit-error rate," says Nadeau.
NIC cards in your PC and the interfaces on your routers in Layer 1 display this information with binary code, in the form of ones and zeros. The binary one represents a "pulse" that shows the data being sent. Custom-made software takes the link test pulse and makes a digital equivalent of it so that this information can be verified. Layer 2 governs data link and media access issues.
One lab, many uses
Vendors can use a pre-defined test procedure from the IOL for conformance or interoperability testing and have the lab conduct the testing. The vendor may be present for the testing, observe, or even have the testing done remotely.
At times, the lab can also be used as a training tool. A vendor will sometimes send new engineers to the testing lab to help reinforce protocol knowledge. Also, the lab provides students at UNH with the knowledge of how to apply the technology and solve real world problems. The lab utilizes students from all areas to create a unique and diverse testing environment, including philosophy and history majors. "Philosophy and history majors can be useful because they will question the standards," says Froning. This creates an environment in which the IOL can evaluate a standard or product from a fresh perspective.
The lab can be used to solve unique situations that a customer may run into. To do so, the IOL will re-create an environment, re-create the problem, and identify what went wrong. At times, vendors will use the IOL as a pre-certification body and ask that the lab evaluate a new design. Contracted services are part of the basic consortium membership. These evaluations are done in a way that allows the vendor to re-create the test on its own so that it may verify the results and make necessary adjustments to the product. Using a number of on- and off-the-shelf testers help to make the results accurate.
Structured membership
Members of the Gigabit Ethernet and Fast Ethernet Consortiums agree to keep a technical focus, elect leaders from its membership, and cover the costs of developing and performing tests through an annual membership fee of $10,000. This can be an inexpensive solution for companies with smaller budgets. Also, members must agree to provide a platform representing the company's equipment at the IOL for at least 18 months. This requirement lets users at the lab perform interoperability testing with current equipment throughout the membership and without having to make any special legal arrangements.
The intent of most of the testing programs of the Gigabit Ethernet and Fast Ethernet Consortiums, and of the lab itself, is to isolate problems in members' equipment before it gets into consumers' hands. Testing is performed from a quality assurance point-of-view, and therefore, the membership agreement limits the disclosure of specific product test results to respective members only.
Ryan Cliche is Assistant Editor of Cabling Installation & Maintenance.