From the May, 2014 Issue of Cabling Installation & Maintenance Magazine
For facilities that simply cannot be offline, specifications prescribe the highest levels of cabling redundancy.
For some data center facilities, the prospect of downtime is a nonstarter. Financial institutions are frequently referred to as fitting this description, because the monetary transactions that take place digitally, i.e. in data centers, (see the book Flash Boys: A Wall Street Revolt by Michael Lewis) are their very business. Data center networks with stakes as high as these concern themselves deeply with issues such as latency, because the almost-infinitesimal duration of a delayed transaction literally can mean dollars lost. And in the global economy, some financial market is always open for business. Under these conditions, downtime equates to business disaster.
But such is not the case for every business. Certainly the reliability of an organization's data center is a paramount concern, and all things considered, every one of these facilities has to be up and running practically all the time. Practically. The extent to which an organization can tolerate data center downtime varies depending on the extent to which the data center is critical to the business's day-to-day, minute-to-minute operation.
Count the cost
In the fall of 2013 Emerson Network Power (www.emersonnetworkpower.com) commissioned a study that was carried out by Ponemon Institute (www.ponemon.org), which sought to quantify the cost of downtime for data centers in the United States. The two organizations combined on a similar study in 2010, so were able to identify changes over the three-year period. In its 2013 study, Ponemon pegged the cost-per-minute of an unplanned data center outage at $7,900. That was up from $5,600 per minute in 2010. The 2013 report analyzed costs at 67 facilities, each of which had at least 2,500 square feet of data center space. It combined direct and indirect costs in its calculations, including damage to mission-critical data, impact of downtime on organizational productivity, damage to equipment, legal and regulatory repercussions, and lost confidence and trust among key stakeholders, Emerson Network Power explained when releasing the study's results in December.
Emerson Network Power also observed, "Those organizations with revenue models that depend on the data center's ability to deliver IT and networking services to customers, such as telecommunications service providers and e-commerce companies, and those that deal with a large amount of secure data, such as defense contractors and financial institutions, continue to incur the most significant costs associated with downtime, with the highest cost of a single event more than $1.7 million."
The role of redundancy
A chief means of preventing downtime is to have redundant systems, including utilities like electricity and water. But data centers also can--and, many argue, should--strongly consider redundant cabling systems within their networks. EPI (www.epi-ap.com) is a data center services company based in Europe, with global operations. EPI provides training, consulting and audit-certification services for professionals who work in and with data centers. Recently the company introduced two courses that certify students to the ANSI/TIA-942 Telecommunications Infrastructure Standard for Data Centers. Students who successfully complete the respective courses and their associated examinations can earn the Certified TIA-942 Design Consultant (CTDC) or the Certified TIA-942 Internal Auditor (CTIA) designation. PEI is developing a Certified TIA-942 External Auditor (CTEA) course and exam, which it plans to complete by the end of 2014.
When interviewed by Cabling Installation & Maintenance, PEI's chairman and chief executive officer Edward van Leent provided insight into the landscape of data center facilities and some general attitudes toward system redundancy. We asked van Leent if redundancy is a high priority across most or all building systems, such as power, water supply, and others. "This is very much a business case," he explained, "and often budget-driven. In a mission-critical data center environment proper care will be given to the design and the various redundancy matters which are directly related to ICT [information communications technology] availability. Aspects such as security, fire detection/suppression are in most cases not designed with full redundancy in place, as failure of such systems do not have a direct impact on the ICT availability. There are cases where legacy data centers were designed at a lower level of redundancy, as a number of years ago this particular data center might not have been so mission-critical yet. However, over the years, this might have changed, and as a result the data center might not meet the redundancy levels now required by the business."
van Leent further described how the business case dictates choices: "Redundancy is typically based on the business case, whereby downtime of the IT infrastructure should be reduced, from a business perspective or even from a government perspective--there may be laws in place where public service providers, such as banks, public transport and others, get penalized if public service provisioning is interrupted. This clearly drives the need to ensure that the services are meeting the needs.
"However, application availability does not always rely on facilities redundancy. Due to the increased network capabilities over greater distance, we see a trend whereby multiple data centers are clustered together to provide high availability from an application perspective, without the need for each physical data center having high levels of facilities redundancy."
Structured cabling
When it comes to network infrastructure, including structured cabling, van Leent indicated that redundancy is quite common. "From the data centers where we have conducted audits and assessments, it is clear that most of the time the network infrastructure has a reasonable to high level of redundancy, even in data centers where the facilities infrastructure has a low level of redundancy. In TIA-942 terms, the network is often rated at level 3 or 4, whereas the power and cooling could be at level 2. Sometimes this is attributed to the fact that investment in IT is often regarded to be a business investment, as everybody understands that IT downtime leads to business downtime. But investments in facilities are often regarded as cost and, as unrealistic as it might sound, not every company understands the fact that if the power has a lower redundancy setup than the ICT setup, there is still a high level of risk.
"Having said that," van Leent reflected, "although the network might have a good level of redundancy, often it lacks proper physical separation between the redundant paths. Obviously, data centers designed specifically for high availability do take both facilities and network infrastructure redundancy--and physical separation--into account."
The TIA-942 terminology van Leent used refers to the annex to the ANSI/TIA-942-A standard that defines levels of redundancy for data center cabling systems. The annex includes four such levels; the higher the number, the greater the redundancy. The figure within this article sketches out the paths required to reach certain redundancy levels.
Until recently, the word used in ANSI/TIA-942-A to describe redundancy levels was "Tier." In mid-March, the Telecommunications Industry Association (TIA; www.tiaonline.org) reached an agreement with The Uptime Institute (www.uptimeinstitute.com) whereby TIA removed the word "Tier" from the 942-A standard. The Uptime Institute established the Tier Classification System in the 1990s as a way of evaluating data center infrastructure as it relates to availability.
The actual changes made to the ANSI/TIA-942-A document stemming from this agreement between the two organizations were minor. The standard never referenced The Uptime Institute to begin with, so no change needed to be made in that regard. And the modifications that were made amounted to editorial changes, not substantive changes. In other words, a data center administrator will do nothing differently now than what was done prior to the editorial change, in order to achieve the levels of redundancy prescribed in the annex to ANSI/TIA-942-A. The TIA reissued the 942-A standard under a new date of March 2014. However, that reissue date does not reset the clock on the ANSI-specified five-year revision cycle for the TIA's cabling standards.
Patrick McLaughlin is our chief editor.
Archived CIM Issues