David Smith, front, and Dave Martinez, behind, work on a server submerged in liquid at the HPC center at Sandia. credit: Craig Fritz
Exploding data center demand for power has brought on the growing adoption of liquid cooling for servers used in compute-intensive HPC-AI clusters. Now, by directly submergining electrical circuits and components in fluid that does not conduct electricity, Sandia National Laboratories researchers are taking liquid cooling to another level with the result, they say, that power consumption can be cut by up to 70 percent.
The researchers say their technique captures 100 percent of the generated heat, almost eliminating the need for the power-hungry fans and chillers used in conventional cooling systems.
An article recently released by Sandia quoted engineer David Damm: “It’s pretty cool — and a little unusual — to walk into a computing lab and see electrical hardware submerged in fluid.” It includes a video with the sight of a technician plugging cables into components sitting in liquid.
Sandia said it is formally testing a commercial nonconductive liquid system from Barcelona-based company Submer Technologies throughout the summer. Early positive results led Martinez to envision a reduction of power and water use that are expected to hinder the development of faster, more powerful machines.
According to the Submer website, their single-phase immersion cooling system uses a biodegradable, nontoxic, nonflammable and noncorrosive liquid. The company says it is 95 percent more efficient than traditional cooling technologies, while 1,400 times a better thermal conductor than air and eight times less electrically conductive.
In 2020, Submer, in collaboration with Intel, introduced the idea of a liquid coolant that would bathe all components of a computing system to provide even cooling and allow higher operating temperatures.
David Martinez of Sandia National Labs
As a bonus, the warmer temperature of the returning liquid can offer some heat to adjacent buildings during winter months and can be used to heat labs and showers in the summer through heat-exchange processes.
Complete immersion of computer components appears to be the most effective way to cool system, said Dave Martinez, engineering program project lead for Sandia’s Infrastructure Computing Services. “We place whole computers — their power cables, everything — in a liquefied solution. We take an entire rack and drop it into fluid contained in big immersion tanks.”
He believes that this direct contact between a liquid coolant and electrified equipment “could reshape the future designs of data centers.”
Unlike water-chill systems that require evaporation to lower operating temperatures, no water is lost; the coolant gives up its heat to the open air, given the right temperature differential, according to the Sandia researchers.
Negotiating through Adacen Inc., a local Albuquerque data company, Martinez convinced Submer to partner with Sandia. Submer would provide fluid and equipment and Sandia would provide its expertise in system cooling and testing to perform a technical evaluation of the system.
In partnership with Adacen, code is being written specifically for the new task by Hoonify Technologies, composed of five former Sandia researchers who have taken entrepreneurial leave from the labs.
Sandia will conduct tests on this system, evaluating hardware reliability and infrastructure support cost and effort. A case study is expected to be released this fall to assess the pros and cons of the technology.
David Smith makes network connections to a computer server submerged in liquid as part of testing at the HPC center at Sandia. credit: Craig Fritz
Martinez said he envisions building the most energy-efficient data center in the world. This center would aim not only to reduce energy consumption but also to use waste heat for various purposes, including heating buildings and preheating lab water.
He leads Sandia’s involvement in COOLERCHIPS, a DOE Advanced Research Projects Agency-Energy initiative aimed at defining the future of liquid cooling in high-performance computing. Additionally, he plays a significant role in Energy Efficient High Performance Computing, a worldwide organization with 1,000 members, where he serves as the infrastructure lead.
Throughout his career, he has witnessed the evolution of computer cooling methods from liquid to air and back to liquid cooling. With a focus on the thermodynamics of computing systems, he has been involved in power and cooling solutions for nearly four decades. Contributions include assisting in the design of energy-efficient data centers, such as the one at the National Renewable Energy Laboratory, which currently boasts the world’s most energy-efficient data center with 80 percent heat recapture.
Speak Your Mind