Quality-assurance techniques may play a big role in network reliability.
Ken Ditto / Telect Inc.
If you are like most people concerned about network reliability, you probably constantly look for ways to make sure your network includes only high-quality components. You undoubtedly spend significant time and effort researching and specifying the network's costly, critical components, with the hope of purchasing the best that the market has to offer. But you most likely do not spend the same amount of time checking out the quality of less expensive components, even though they, too, have the ability to bring your network to a screeching halt.
We often view these inexpensive elements as commodity products-available from many sources, fairly inexpensive, and easily replaced. We assume the manufacturer has a good quality-control program in place, and the products we purchase are free of defects. Operating under these assumptions, we generally take the performance of these products for granted, never really giving due respect to the reality that the failure of a $20 component can be just as devastating to a network as the failure of a more costly piece of equipment.
Key issues worth examining
In light of the impact these commodity products can have on the network, several key issues are worth examining. First, what type of quality-control processes do the manufacturers of these inexpensive components use? Second, does it make sense to seek out higher-quality commodity products when they perform a critical function for the network?
Let's look at these commodity components from a different angle, to give us some perspective on how these products' value can vary depending on the role they play in a network. Suppose I am a guide who takes vacationers into the mountains on horseback. Part of my job is to arrange for the supplies, to assure my clients have a great time and are well fed. What happens if, during the first meal, someone breaks one of the plastic knives I provided? What impact does this have on our trip? Probably not much. As a conscientious guide, I most likely packed extra plastic utensils. So, all the client has to do is ask for another.
In other words, this commodity product-the knife-was easily replaceable.
What would happen if I didn't pack extra knives? It still probably would not be a big deal, as the client would simply use something else as a substitute. In this case, the commodity product did not have a critical role to fulfill, so its failure was not a big problem.
But what would the impact be if the client also expected to use that knife to sharpen sticks for roasting food or cutting firewood? As the utility of a product becomes more important to the success of a mission, and easily obtainable substitutes are less available, the product requires closer attention.
In the case of the knives, there is an excellent chance that I, as the guide, would be more selective in the purchase process if they were critical to the excursion's success. Perhaps I would even purchase a different type of product, like choosing a metal knife instead of a plastic one, even though the metal knife is more expensive. If a knife is critical for the mission's success,
I might even purchase a high-quality knife with a heat-treated blade and protective cover, and perhaps even tools and supplies to maintain its integrity, sharpness, and usefulness. I might even train the vacationers how to use and care for their knives.
What does any of this have to do with network reliability?
In simple terms, many elements of a communications network can be compared to the knife. If we view network components as commodities, and regard them as easy to replace or substitute without substantial cost when they fail, we may not spend much effort to ensure we have high-quality products. But as an individual network component becomes more critical to the reliability of the entire network, we should put more time and effort into the purchasing, maintenance, and training pro-cesses to ensure these products meet a certain quality level.
For the horseback excursion, would we carefully inspect the plastic knives before the trip if I considered them to have little utility? Probably not. Instead, we assume someone has done at least a cursory inspection to make sure the product meets the standards necessary to accomplish the job. But if our survival in the wilderness depended on how well the knife held up during use, you can bet that quality would be a key concern.
Some operators of local-area and wide-area networks (LAN/WANs) make similar assumptions regarding network components, especially inexpensive products, assuming they meet a certain level of quality and the manufacturers adhere to industry standards. Unfortunately, these users often end up paying a high price later when a commodity product causes a network failure. Network operators sometimes learn through costly failure that even though the component is relatively inexpensive, it is critical to the mission's success.
Statistical sampling
Staying with the plastic-knife example just a little longer will help shed light on the quality-control methodology used today by many network-component manufacturers. It is probably not reasonable to expect that the plastic-knife manufacturer will carefully inspect and test every product that rolls off the assembly line because most consumers place more value on the purchase price of the product than they place on the product's quality. It costs money to test every single knife, so individual inspections would, therefore, raise the product's price. When purchasing plastic knives, have you ever carefully inspected all the brands available at the grocery store, then purchased one based on quality? If so, you are in a distinct minority. Most consumers pick the lowest-cost product.
The plastic knives' quality most likely was checked using some type of sampling process. Perhaps an inspector randomly selected a certain number of knives and examined them. The inspector probably had a list of parameters the knives had to meet, along with measurements and dimensions. The inspector may have looked at the finish quality and maybe even bent the knives to the point of breaking to measure the product's strength. If all of the selected knives passed inspection, the inspector probably assumed all knives were acceptable. If some failed, the inspector may have taken another sample. If they all failed, the inspector probably stopped production to locate the problem.
In many cases, the same type of quality-control process is used to monitor the quality of inexpensive, yet critical, optical-network components. And unfortunately, recent evidence suggests that these methods do not adequately ensure a high level of network reliability. That being the case, many network administrators must be much more concerned about quality issues for so-called "commodity" components.
Statistical process control (SPC), statistical sampling, or simply sampling are all terms for similar quality- or process-control methods used by many manufacturing companies, including those making plastic knives and network components. Both the implementation of and results from this method vary widely. Companies can claim the use of statistical sampling by employing such simple processes as measuring certain parameters and averaging them together.
On the other hand, SPC can be as comprehensive as automatically collecting data, providing the trend results in real time, updating appropriate databases, and determining if the process meets what is known as "six-sigma" methodology-the equivalent of 3.4 defects per million parts, or 99.9997% success.
So, within the industry, there is a wide range of inconsistency when it comes to understanding which parameters a manufacturer is actually implementing when using SPC. For network users, the most important issue is SPC's impact on product quality, which can be significant or minimal, depending on the implementation. The SPC methodology defines precisely how many defects are acceptable for a certain volume of products, but even one defective product is unacceptable when it ends up installed in your network.
Buying from commodity specialists
A real-life example of a commodity product in a LAN application is a fiber-optic cable assembly, which connects the termination equipment with the cabled portion of the network. It has a connector on one end and a bare fiber on the other end that is typically spliced, either mechanically or through fusion, to the fiber-optic cable.
Generally, network operators purchase cable assemblies from a manufacturer, which installs the connector on the cable and polishes it so that a high-quality connection is possible. Network operators occasionally field-terminate the cable, but this process takes time and the results usually are not as consistent as those achieved through factory terminations. Compare this situation to purchasing plastic knives, as opposed to carving the knives out of wood just before you eat. You can do it, but there are some compelling reasons to purchase from suppliers that specialize in producing specific components.
Network operators expect purchased fiber-optic cable assemblies to have certain attributes. For example, it is reasonable to assume the termination process has been carried out in an environment designed to achieve successful results, and that properly trained technicians carry out the process. Purchasers may also expect that certain tests have been performed on the assembly to ensure it meets industry standards. Other expectations may exist regarding such attributes as the quality of the components used in the assembly and how the assembly has been handled to avoid damage. Additionally, network operators usually assume a manufacturer has a good understanding of the required processes necessary to meet the applications of a LAN environment. But these remain assumptions until they are documented.
A connector's true worth
So, how does the network operator know the fiber-optic connectors he or she purchased will perform as expected? And should this be a major concern? Cable assemblies are inexpensive pieces of the network, typically costing about $20 apiece. Users can purchase them from at least 100 different sources. And as long as the assembly is a somewhat common configuration, replacement parts can generally be delivered within 24 hours. Plus, the network operator probably has spares in inventory. Given all these factors, connectors are often viewed as typical commodity products. Why worry about the quality of a cable assembly?
First, consider the consequence of a network failure. If one link in your LAN fails, what is the real result? How many users are interrupted? To estimate the cost-per-hour of downtime, multiply the total burdened cost of each user by the number of users that would be interrupted. If you can place a dollar value on potentially lost data, compute that as well. Other questions to ask about network failures include: Will interrupted processes have to start over from the beginning? Will your customer service suffer? Are customers inconvenienced? Do you lose business because you are not able to respond?
Once you establish an estimated value per hour for a downed network, estimate how long it will take to identify the problem. When your network fails, do you always know exactly where the problem is? How long has it taken you to locate the source of past network failures? After you identify the failure source (in this case, the optical cable assembly or connector), how long will it take to replace it? Considering all these factors, you can estimate the cost of having a network failure due to a poor-quality optical connector. This thought process certainly puts a different level of importance on the inexpensive connector.
As for how a network operator can know if purchased optical connectors are adequate for a given application, Telcordia Technologies (Morristown, NJ), and the Telecommunications Industry Association/Electronic Industries Alliance (TIA/ EIA-Arlington, VA) have developed optical-connector requirements. The two standards are in close agreement. Telcordia outlined specifications for optical connectors in GR-326, and has provided details of the specifications as well as the tests an optical cable assembly should pass. The 189-page document is comprehensive regarding the requirements for insertion loss, reflectance, durability, endface geometry, remateability, and other characteristics.
By choosing products that comply with industry standards like the TIA/EIA or Telcordia specifications, network operators can make great strides toward ensuring only quality components-in this case, optical connectors-are put in service. Confirming that vendors comply with industry standards, however, can be difficult.
Investigate the process
The most common method vendors use to measure industry compliance is to test a sample of the manufactured parts for conformance to the requirements. In the case of low-cost cable assemblies, for example, most manufacturers consider it too costly to measure every component. This sounds a lot like the SPC methodology used for plastic knives detailed earlier.
If a sufficient quantity of samples is measured, the data will form a normal curve in accordance with what is known as the Central Limit Theorem. This normal curve can serve as a measure of the process, and the process can then be determined to be capable or incapable. "Capable" means the process produces the desired result, and "incapable" means it does not. Essentially, a process is capable when the entire normal process distribution falls between the upper and lower specification limits.
In a capable setup, there is a very small probability that any manufactured product will be outside the specification. As the distribution curve becomes narrower, indicating less variation because of process improvement, the probability of a product's not meeting the specification decreases as long as the specifications do not change. It is possible to improve some processes to the point where it is extremely unlikely that any product would not meet the specification. The question that arises from this statistical-model analysis is: How does your vendor's process measure up against the specifications you require?
Other elements also help determine manufacturing processes' capabilities, and the conformance of products to industry specifications. For instance, to be valid, the data samples must be random and statistically significant for the volume and degree of probability desired. Additionally, to adequately affect the process, the results must be available in a timely manner. It does little good to discover that last month's shipments had a high probability of failing to meet specification.
So, to determine if your vendor provides quality components that meet industry standards, it is necessary to determine if the vendor uses SPC, and then how the vendor implements the SPC process. A sampling process implemented to a high degree, such as six-sigma, will produce products that are defect-free 99.9997% of the time. On the other hand, a less-strict process implementation can allow a larger number of defective products to reach the market, and potentially your network.
Failing to make the grade
Even if a manufacturer uses some type of SPC to monitor quality, the product still may not meet industry standards. In an effort to determine the level of compliance to industry standards, Telect Inc. (Liberty Lake, WA) purchased optical cable assemblies from five manufacturers that claimed to meet Telcordia's GR-326 Issue 3 standards. In all probability, these tested cables came from manufacturers using SPC technology. That is an assumption we made, not a confirmed fact.
An independent laboratory tested each cable for conformance to the specifications for insertion loss, reflectance, endface geometry, and endface quality. In spite of each manufacturer promoting its products as industry-compliant, test results indicated a high probability that some manufacturers are selling products that do not meet all the specifications.
Results of the test for fiber height-one of a series of tests that determine endface geometry-indicate a reasonable possibility that three of the five manufacturers will produce optical terminations that do not meet the specification they claim to meet. Using connectors that do not meet the fiber-height specification can negatively affect network performance. For example, a connector with a too-high fiber height can damage the optical interface of the mate connector. One with a too-low fiber height can fail to make physical contact with the mate connector, thereby increasing reflectance and insertion loss.
In each case, the results of the other tests showed that some manufacturers did not meet the industry-standard requirement. In fact, only one of the five manufacturers tested had 100% of its cables meet the required specifications. These tests imply that simply choosing a vendor that claims to provide industry-compliant optical cable assemblies does not necessarily ensure you will not experience network failures due to faulty connectors.
Are standards strict enough?
Also consider that industry standards may not specify everything required for network reliability. Telcordia's GR-326 document does not provide detailed specifications for the endface quality of optical connectors. The independent testing of optical cable assemblies mentioned previously showed that endface quality varied widely among manufacturers.
Any defects on the end of a fiber can instantly or eventually affect network performance. Under current industry standards, a connector with scratches, pits, and a rough epoxy ring could still be deemed acceptable, because the standards in this area are vaguely defined. If the defect is not obstructing the fiber core, where light is carried, and it does not impede alignment, it may have no initial adverse effect. As time passes, however, pressure from the connector springs and various environmental elements (such as temperature variations, humidity, movement, and vibration) may alter the endface. As a result, defects may grow to a magnitude that interferes with optical performance.
By definition, surface defects are interruptions in the smooth surface of the glass, and these interruptions can migrate under pressure. Research shows that even the movement of air on the bare surface of glass can cause surface imperfections to grow. Studies show that the tensile strength of fiber decreases when surface imperfections are exposed to the environment. Further research concludes that defects in optical fiber can propagate when exposed to humidity, vibration, and temperature swings.
Surface defects can also become reservoirs of moisture and trap other contaminants. Moisture is particularly harmful because of absorption, which reduces mechanical strength. Other types of contamination may abrade the surface, causing scratches, pits, and chips. One manufacturer has even suggested that certain contaminants have an affinity for the light-carrying area of the fiber. Any contamination in the optical path can degrade the reliability of the transmission.
Although several models have been developed to predict reliability, the severity, nature, and time of failure due to surface defects are largely unpredictable. Because these defects demonstrate the likelihood of changing, network failure becomes a real possibility. Empirical data and scientific research imply that eliminating surface defects minimizes network failures due to connector quality. Even though GR-326 does not adequately deal with this performance area, it merits special attention to anyone concerned with network reliability. As a network operator, you may discover other critical elements that are not necessarily specified by current standards.
More than random sampling
Ensuring that only high-quality components make it into your network requires some investigating, but the results could eliminate costly network failures. Using optical cable assemblies as an example, one way to make sure your cables meet specifications is to inspect each one. The drawback to this practice is that it requires expensive equipment, like interferometers, precision power meters, and reflectance testers; it also requires trained operators and time.
Another approach to guaranteeing quality is to develop an in-depth understanding of the processes that your vendors use to produce quality parts. For instance, how do they implement their SPC programs? Do they integrate sophisticated software with their data sampling? Are they working toward a six-sigma level of performance? How do they qualify the parts used in their products? What is the corporate philosophy regarding quality? Are their manufacturing practices consistent?
A reputable vendor will gladly discuss these areas of quality control with its customers.
A third approach to ensuring quality network components is to purchase products from a vendor that inspects and tests each part for compliance to all critical specifications. A 100% inspection policy leaves little room for process interpretation. This method increases the cost of manufacturing by 10% to 30%, but can also result in increased network reliability. In the optical cable tests mentioned previously, the only manufacturer that passed all of the tests employed 100% testing and inspection to assure quality.
One final consideration for ensuring network reliability is adequate training of personnel regarding component installation and handling. Even the best optical connector's performance capability can diminish dramatically if it is not clean. Network technicians should have a clear understanding of how to properly place a component into service, how to maintain it, and how to store it properly. A little training can go a long way toward maintaining performance.
Network operators have a lot riding on their investments, and they face enough challenges without having to worry about failures due to poor component quality. Simply assuming all purchased components meet quality standards is risky today, given the high cost of a network failure. Many types of quality-control methods commonly used on network components simply do not keep all defective products from the marketplace. By paying careful attention to each component that is critical to reliability-including "commodity" products-network operators can have confidence their systems will provide the necessary service.
Ken Ditto is fiber-optic product manager for Telect Inc. (Liberty Lake, WA). You can reach him via e-mail at [email protected].
Cabling Installation & Maintenance Story Ideas
Do you have an idea for a story on system design, installation, maintenance, or troubleshooting? Send it to:
Arlyn S. Powell, Jr.,
Group Editorial Director
Cabling Installation & Maintenance
98 Spit Brook Rd., Nashua, NH 03062-2801
tel: (603) 891-9215, fax: (603) 891-0587
e-mail: [email protected]