Applications for Infrared Thermography at Computer Centers

By Eric Stockton CompuSCANIR™, a division of Stockton Infrared Thermographic Services, Inc. 8472 Adams Farm Road Randleman, NC 27317

http://www.compuscanir.com
800-248-SCAN

ABSTRACT

Downtime at Computer Centers is not an option! The equipment is expensive and even more important, the downtime created by a failure can be extremely costly – much more than the average office building or factory. Electrical and mechanical infrared predictive maintenance (IR/PM) is a must for the safe, continuous operation of the facility and the steady flow of information. The electrical switchgear, UPS, ATS and Server Systems should be checked with infrared thermography and other testing means on a regular basis. These are specialized systems and the infrared thermographer needs to be familiar with the systems and with the operation of the center. This paper describes the need, different applications, methodology and equipment required to perform IR surveys at computer centers.

PERFORMING THE IR SURVEYS

Computer centers have normal loads like any other building that feed the non-critical items in the building, but the data systems are specialized systems. These are:

  • Dual-power technology requires at least two completely independent electrical systems. These dual systems supply power via diverse power paths to the computer load by effectively moving the last point of electrical redundancy from the uninterruptible power supply (UPS) system downstream to a point inside the computer hardware itself. Mr. Kenneth Brill’s intuitive conclusion has since been confirmed by The Uptime Institute’s research that has determined that 98% of all site infrastructure failures occur between the UPS and the computer load.

  • Resistive loads are used to fully simulate and test all equipment before it is put on-line on the computer floor. Any problems that are encountered must be completely repaired and the system rechecked fully before putting the equipment on-line.

  • All panel covers (as well as the dead fronts) need to be removed before the test, so pertinent components may be evaluated under load. Risk managers may resist removing panel covers and dead fronts to complete the infrared survey, so they must be in agreement that this is necessary.

  • Automatic transfer switches (ATS) must be checked on the Emergency Source (usually a back-up generator) as well as the Normal Source.

  • Much of this work must be done during “maintenance windows”-a time where the usual transfer of data is reduced or eliminated, usually a Sunday morning (Midnight until 8 AM).

  • Battery back-up systems must be checked in a real-time battery discharge situation to fully simulate an actual loss of the Normal Source as well as the Emergency Source.

  • There must be a total accountability of all infrared survey results, especially all of the equipment associated with the UPS and computer/server systems. This can be accomplished by recording the entire survey on videotape and/or snapping fully-radiometric images of all equipment whether problems exist or not. In either case a data log of all equipment surveyed must be created including a time/date stamp reference for all equipment.

  • Uniform cooling of all data center server, storage, and computer equipment is essential for proper operation. The American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. has provided Thermal Guidelines for Data Processing Environments as a reference.

EXAMPLE THERMOGRAPHIC FINDINGS ON COMPUTER CENTER EQUIPMENT

Figure 1. Thermograph and photograph showing lug connection on UPS Module Inverter Assembly.


 

Figure 2. Thermographs showing heat distribution patterns on a server racks.


 

Figure 3. Thermograph and photograph showing fuse lug connection on an UPS Module.


 

Figure 4. Thermograph showing fuse lug on UPS Module input filter fuse.


 

Figure 5. Thermograph showing overloaded breaker on a power distribution unit (PDU) on the computer floor.

 

Phase Left Side Amps Right Side Amps Total Amps % Load Normal Max 275 Amps % Load Two Feeder Max 525 Amps
A 69.8 129.5 199.3 72% 38%
B 195 1.8 196.8 72% 37%
C 193 2 195 71% 37%

Table 1. Measured current on inverted Y 23 KV oil circuit breaker line side switches, each leg and total.

OTHER CONSIDERATIONS

The computer room floor cooling system must be designed and engineered to provide uniform cooling of all computer components. The objective of the design of the cooling system is to provide a clear path from the source of the cooled air to the intakes of the servers. The heat from the equipment must also be efficiently returned to the return air duct of the CRAC (Computer Room Air Conditioning) Unit. Infrared thermography can be performed to ensure proper and even cooling of the room.

All Mechanical Systems have the same stringent requirements that the electrical system. This is achieved by redundancy and failure prevention engineering. The Federal Reserve Board (Board), the Office of the Comptroller of the Currency (OCC) and the Securities and Exchange Commission (SEC) are publishing an Interagency Paper on Sound Practices to Strengthen the Resilience of the U.S Financial System. The paper identifies three new business continuity objectives that have special importance in the post-September 11th risk environment for all financial firms. The paper also identifies four sound practices to ensure the resilience of the U.S. financial system, which focus on minimizing the immediate systemic effects of a wide-scale disruption on critical financial markets. The agencies expect organizations that fall within the scope of this paper (like infrared) to adopt the sound practices within the specified implementation timeframes, as described in more detail in the paper.

SUMMARY

Computer Data Center Systems Require 100% Uptime. To achieve this, it is essential that the thermographer:

  • Record all IR work and be accountable for the complete infrared survey.
  • Be expert in the IR applications of computer equipment.
  • Be knowledgeable of the different test scenarios and equipment operability.
  • Be willing to work “off-shift” on weekends, etc.
  • Understand how to work with the electricians and computer technicians so that the equipment can be tested with a resistive load in a real-time situation at 100% expected loads.

REFERENCES

1) W. Pitt Turner IV, P.E., John H. Seader, P.E. and Kenneth G. Brill; “Industry Standard Tier Classifications Define Site Infrastructure Performance”; pp. 1-4; 2001-2005 The Uptime Institute, Inc.

2) “Thermal Guidelines for Data Processing Environments”; 2005. The American Society of Heating, Refrigerating and Air-Conditioning Engineers Inc.

3) “Interagency Paper on Sound Practices to Strengthen the Resilience of the U.S. Financial System”; U.S. Securities and Exchange Commission.