6 September, 2018
Will Standard Cooling Methods Suffice in High-Density Designs?
In this blog, we’ll explore whether standard cooling infrastructure designs utilizing simple cooling capacity and airflow requirement calculations are appropriate for very high-density facilities. We’ll use an example model to demonstrate the importance of understanding airflow delivery when dealing with such a scenario, and conclude with a proposal that virtual modeling is the only cost-effective and viable method of doing so.
Digital Twin Model of a high-density data center in 6SigmaRoom
The digital twin model we’ll use for this example includes a cooling capacity that is more than twice that of the installed IT.
Data center rack-level power definition plot
The room details are as follows:
Baseline Simulation Results
The baseline model’s overheat plot shows the need for a well-designed cooling delivery strategy when operating a high-density data center. In this case, despite the over-provisioned cooling infrastructure, only 25% of the power load is being sufficiently cooled.
Baseline model rack-level overheat plot
The simulation results highlight many areas where the cooling strategy’s effectiveness could be altered, but the most obvious issue arises from the control system. Using return temperature control is dangerous in many capacity loading scenarios, and high-density loading compounds this problem. While some of the CRAC units are running at maximum cooling capacity, others are only running at 10%; this is due to a huge variance in return temperature that originates from asymmetrical room loading. As a first step, we need to ensure that the control system is not inhibiting the cooling infrastructure’s capabilities. The simplest way to achieve this is by switching to a supply control strategy.
The supply control strategy provides an immediate improvement, but it is still insufficient: 45% of the power load is now being sufficiently cooled. On closer review of the simulation results, we can see that many of the issues lie within the higher-loaded racks in the center of the room. This suggests that there are airflow issues causing recirculation in these regions. To combat this, there are two effective methods that can be considered: hot and cold aisle containment. The digital twin allows us to test both methods for resilience before any deployment decisions are made.
Supply control rack-level overheat plot
Supply Control Model - Recirculation
Recirculation over and around racks in an overheating row
As expected, there is a large amount of recirculation occurring in this row. A deficiency between the IT airflow requirement and the floor grille flow rate causes low-pressure regions in the cold aisle. This results in air being drawn over, through and around the racks, which increases inflow temperatures. Installing containment is an effective way to combat this recirculation. While this does not rectify the flow deficit, the recirculation flow path above and around the racks is blocked so the amount of hot air reaching the IT inflows is reduced. Again, the effect of this installation can be simulated and analyzed.
Cold Aisle Containment (with supply control)
Cold aisle containment rack-level overheat plot
Looking at the simulation results, it’s immediately clear that containment increases the effectiveness of the airflow delivery: 95% of the cooling load is now being sufficiently cooled. Adding cold aisle containment separates the cold and warm airstreams, which reduces mixing and increases efficiency. However, this reduced mixing results in higher air temperatures in the IT exhaust regions, so even a small amount of recirculation through well-sealed containment and rack blanking could result in overheating IT equipment. This issue is apparent in all forms of containment. Due to the over-specification of cooling infrastructure in this room, another form of containment should also be considered: a flooded room, hot aisle containment scenario.
Hot Aisle Containment (with supply control)
Hot aisle containment rack-level overheat plot
Installing a hot aisle containment system requires more retrofitting than the previous scenarios, including CRAH unit ducting and a dropped false ceiling. However, this solution results in a sufficiently-cooled room, with all the IT equipment receiving air 9F or more below the critical temperature of 100F.
The results highlight the ineffective nature of designs using simple heat addition and rejection calculations. These calculations miss vital details: while at room level the cooling strategy can satisfy thermal limits, without modeling it is impossible to know whether this is reflected at cabinet or IT level.
The digital twin made it possible to install a 2MW load in a facility with less than 9000sqft of white space. While many modeling caveats must be considered when applying this methodology to a real scenario, it is undeniable that the digital twin gives engineers a scientific foundation to build from without requiring costly testing or inaccurate assumptions. Following these compliance-centered simulations, you can use the digital twin to explore different ways of increasing the facility’s efficiency. It should be noted that installing containment, either hot or cold, is a significant investment and should not be considered as a panacea for airflow issues. Additionally, the simulations described here do not include redundancy considerations; you can easily investigate these scenarios in the digital twin by switching off the most highly-loaded CRAH units.
Blog written by: Mike Eccles, Consultant Engineer
Other Recent Posts
New EnergyStar Option for Defining Airflow
In Release 13, we introduced two options for defining airflow through servers based on data from AS…
How effective is your cooling system?
If one of your air conditioning units (ACU) fails, which part of your data center will be most affe…