Oak Ridge National Lab (ORNL): Oak Ridge Leadership Computing Facility
The Oak Ridge Leadership Computing Facility at Oak Ridge National Lab (ORNL) is home to Summit, a high-performance computing machine. After its launch in 2018, Summit was recognized as the world’s fastest supercomputer. With the capacity to perform 200 quadrillion calculations per second, not only is Summit more powerful than its predecessor, it is also more energy-efficient.
A building-wide retrofit to the 9,400-square-foot data center housing Summit started in 2015. During Summit’s installation, engineers focused on developing an energy-efficient cooling system that could match the supercomputer’s 13 megawatts of energy consumption. Ensuring optimal cooling for a data center is a top priority for operators as it lowers operational expenditures, reduces the strain on equipment cooling mechanisms, and further extends the lifespan of hardware.
MoreMoving away from traditional air cooling with chilled water, Summit’s data center features an innovative, warm-water cooling system, reducing the use of chilled water and the reliance on inefficient chillers. Compared to air cooling, liquid cooling has a higher ability to capture and remove heat. Once excess heat has been transferred to a liquid, it can be removed from the data center more efficiently.
MoreDue to space constraints, a single piping system was selected to serve both the cooling and heat exchanger processes. Water runs through the piping at a temperature of 70°F allowing for chiller-less cooling throughout most of the year. Water flowing through the system’s cold plates captures excess heat from each server’s central and graphic processing units. Residual heat is then captured by rear-door heat exchangers (RDHx). Hot air is forced from the server-rack through the RDHx device where excess heat is exchanged from air to water. The wasted heat that has been transferred to water is then moved to a cooling plant before the process starts over.
Some of the other energy efficiency measures implemented include:
- LED lighting and timer system
- Epoxy-coated pumps to increase pumping efficiency
- 480V racks to prevent excess electrical transmission losses
- Variable frequency drives (VFDs) on motors to align cooling capacity to load
- Variable refrigerant flow system that filters and dehumidifies the room air while removing any residual heat released into the room from IT equipment if needed
Looking ahead, facility engineers project that as Summit enters full production its PUE will continue to improve, likely leveling off at an average below 1.10. So far, the upgrades have led to an annual cost avoidance of $430,000.
LessFor this project, ORNL’s design and construction team received the Tennessee Chamber of Commerce and Industry’s 2018 Energy Excellence Award for their leadership advancing energy efficiency solutions, while designing the world’s fastest high-performance computing system. In addition to the energy and cost savings, the cooling towers are much easier to maintain compared to the previous chillers, reducing operation and maintenance requirements (both in time and cost).
More