22 May, 2017
BayCare Health System, a major healthcare network in the Tampa, Florida area, operates a data center facility. Hoping to reduce inlet temperatures, they contacted Eaton (a global provider of airflow management solutions) to trial a cold aisle containment system (CACS). Initial data showed improvements to some IT, but surprisingly the results were inconsistent with some IT receiving air at higher temperatures. To understand why the CACS had not behaved as intended, Eaton partnered with Future Facilities Ltd (FFL) to investigate - using the 6SigmaDCX engineering simulation tools. We used detailed measurements of the on-site environment and information provided by Baycare to produce a model of the data center. Eaton discovered that the 6SigmaDCX simulations matched the measured values closely.
The data center used Liebert DS105s CRAC units with an airflow rate of 14,600 CFM. Measurements with a balometer (flow hood) showed that the total airflow supply though the floor grilles was 7770 CFM, while the demand from the IT was determined to be 10,500 CFM. This was consistent with the simulated airflow behavior. These results indicated that there was not enough air delivered through the floor grilles to meet the airflow demands of the IT. To respond to this problem, Engineering Simulation and on-site calibration proved to be essential, as it allowed us to study the dynamics of the system.
Building a Data Center Model
The data center facility was divided into three regions in a non-partitioned room. As a trial, the CACS was applied only in Region A, which used 80 Eaton cabinets with Class 1 EIA-310 rack-mounted server equipment. Several cabinets contained equipment with side mounted vents, while blanking was inconsistent and often incomplete. We used 6SigmaDCX to provide an accurate representation of this internal server-rack configuration and blanking provision. This type of modeling is essential to determine inlet temperatures of IT equipment. Baycare provided documentation that mapped the location of the devices in each rack. We used 6SigmaDCX to intelligently match this data with corresponding objects in the software’s library and place them in the correct locations in the model. 6SigmaDCX’s library objects are detailed models developed from the manufacturer’s specifications. The region with the containment system installed was simulated as a virtual facility in 6SigmaDCX. We avoided modeling the entire data center by using measured data to create appropriate boundary conditions for the interface between regions.
To understand the changes produced by using CACS, we simulated the behavior of the room with and without CACS. 6SigmaDCX allows you to trace the airflow in a room by adding ‘streamline plots’ to objects in the model. These can be a useful diagnostic tool.
Simulation Results: No CACS
Figure 1 shows the streamlines tracing cold air flowing out from the floor grilles. The IT at the top of racks and on the end of rows receives little cold air from the air conditioning unit. A large proportion of the cold air is drawn into the rear fan doors of the central racks – leaving it unavailable for IT cooling.
Figure 1: Plot from floor grilles without cold aisle containment
Figure 2 shows the streamlines tracing the path of air flowing into all the equipment inlets. The make-up air (air not delivered through the grilles) is composed of ambient air (light blue), warm remixed air recirculated overhead from behind the cabinets (yellow and green) and direct hot air exhaust through leakage areas within the cabinet (red).
Figure 2: Plot from equipment inlets without cold aisle containment
Simulation Results: Using CACS
Figure 3 shows the cold air flowing out from the floor grilles. Cold aisle containment prevents end of row air recirculation, allowing the cold air from the grilles to reach more equipment at the end of the rows. However, air supply to the top of racks is still limited, and much of the cold air is drawn through the fan doors.
Figure 3: Plot from floor grilles with cold aisle containment
Figure 4 shows the air flowing into the equipment inlets. Using CACS has prevented overhead recirculation and ambient air from entering the containment field. However, since there is insufficient airflow from the floor grilles, the IT is drawing hot air from leakage areas. This is the cause of the increased inlet temperatures.
Figure 4: Plot from device inlets with cold aisle containment
The simulation results show that applying CACS produced some of the desired changes, but these were undermined by leakage and poor airflow. Understanding how the airflow works allowed us to recommend changes that would realize the full potential of the cold aisle containment solution:
• Turn off rear fan doors or replace them with perforated, vented doors
• Seal racks at the bottom and top, and fill in all side mounting rail gaps
• Increase the airflow supply by replacing perforated grilles with grate grilles and enhancing the CRAC units
As a result of these improvements, inlet air temperatures for all front-to-back flow equipment was reduced to within 1ºF of the supply temperature. This allowed the CACS solution to be rolled out across BayCare Health System’s data center.
Blog written by James Barros, Technical Writer
Other Recent Posts
Why Data Center Operators Need to Rethink the Rule of Thumb Practice
Many of us may feel we’re winning the battle against over-provisioning of data center resources, b…
Digital Twins and the New Space Race
The typical data center is currently spending 136% of the OpEx budget required to meet business dem…