NREL's Energy Systems Integration Facility

Background

The U.S. Department of Energy’s National Renewable Energy Laboratory (NREL), located in Golden, Colorado, is renowned for its commitment to green building construction and for leading by example. NREL built the Energy Systems Integration Facility (ESIF), a 182,000-square-foot LEED Platinum research facility that includes high bay and laboratory space, office space for about 220 staff, and an ultra-efficient high-performance computing (HPC) data center.

The data center features a chiller-less design, direct warm-water liquid cooling, and waste heat capture/re-use. It operates at an annualized average PUE rating of 1.06, making it the world’s most energy-efficient data center. This data center demonstrates technologies that save energy and water, reduce CO2 emissions, and capture/re-use waste heat with an estimated annualized average Energy Reuse Effectiveness of 0.7.

Solutions

High Voltage Source

Several key design specifications have led to the data center’s unprecidented efficiency. First, high-voltage electricity (480VAC rather than the typical 208VAC) is supplied directly to the racks, which reduces the need for power electronics equipment and power conversions, and eliminates electrical transmission losses.

Liquid Cooling

Secondly, the data center uses warm-water liquid cooling supplied directly to the server racks. The decision to use liquid cooling was made several years ago, before liquid-cooled systems were routinely available. There are several advantages to this approach. Water as a heat-exchange medium is three orders of magnitude more efficient than air, and getting the heat exchange close to where the heat is generated is most efficient. Also, 75°F water for data center cooling allows the data center to use highly energy-efficient evaporative cooling towers, eliminating the need for much more expensive and more energy-demanding mechanical chillers, saving both capital and operating expenses.

Deeper Opportunities

The ESIF data center is designed and built to capture the “waste” heat generated by the HPC system for heating within the building. By focusing on direct liquid cooling, the energy needed to cool the computer systems decreases dramatically, as does fan energy use. The data center is heavily instrumented to continually monitor data center efficiency and to provide real-time optimization of energy usage.

Savings

When the data center is fully built out to its initial 5 MW capacity in 2018, the NREL team predicts that they will spend $1.2M less on utility costs per year than a state-of-the art efficient data center with a PUE of 1.3. Further savings in avoided utility costs will be realized as the data center expands to a planned 10 MW load in 2022.

Other Benefits

Aside from savings due to avoided utility costs in the data center infrastructure, the highly efficient data center has also led to capital and building operational savings.

Capital Savings

The data center is similar to a typical data center except that it utilizes more efficient, less expensive evaporative cooling towers rather than more expensive, energy-demanding mechanical chillers. Not purchasing the mechanical compressor-based chillers saved approximately $4.5M in capital expenses.

Operational Savings

By capturing the computer waste heat directly in liquid and pumping it throughout the ESIF, the data center serves as the primary heat source for office and laboratory space within the facility. Data center waste heat is also used to heat glycol loops located under an adjacent plaza and walkway, melting snow and making winter walks between buildings safer. Utilizing waste heat for office and laboratory space allows NREL to offset another $200K per year in heating costs.

ANNUAL ENERGY USE

ANNUAL ENERGY COST

ANNUAL WATER USE

ANNUAL WATER COST

ANNUAL PLANT ELECTRICITY USE

ANNUAL PLANT ELECTRICITY COST

IN BUILDING ENERGY USE

IN BUILDING ENERGY COST

ACROSS NETWORK USE

ACROSS NETWORK COST

ANNUAL EMISSIONS

USE

COST

Peregrine High Performance Computer (HPC)

Energy Systems Integration Facility

Evaporative Cooling Towers

Mechanical Equipment Room

NREL's Energy Systems Integration Facility

Related Content

Peregrine High Performance Computer (HPC)

Energy Systems Integration Facility

Evaporative Cooling Towers

Mechanical Equipment Room