Argonne National Lab’s Theta
Scientists at the U.S. Department of Energy’s (DOE) Argonne National Laboratory have conducted what they claim is the largest simulation of flow inside an internal combustion engine. Insights gained from the simulation – run on 51,328 cores of Argonne’s Theta supercomputer – could help auto manufacturers to design greener engines.
A blog post on the Argonne site explains that the simulation is intended to provide a detailed view of turbulent flow, enabling automotive manufacturers to evaluate several potential engine designs. Performing simulations at such a large scale requires compute-intensive resources offered by the Theta system at Argonne’s Leadership Computing Facility. Theta, currently ranked 34th on the Top500 list of the world’s most powerful supercomputers, is an HPE/Cray 11.69-petaflops system based on Intel Xeon Phi processors and Intel Aries interconnect technology with a Lustre-based parallel file system, integrated by Cray’s HPC software stack.
About 18 months ago, research scientists at the Center for Transportation Research in Argonne’s Energy Systems (ES) division decided to conduct direct numerical simulations (DNS) to resolve turbulent flow scales inside an engine. Before that simulation could be performed, however, smaller simulations were necessary to ensure that the largest-ever would go as planned.
“This is one of the most detailed simulations ever of the flow in an internal combustion engine,” said Muhsin Ameen, a research scientist at Argonne’s Center for Transportation Research in the ES division.
Ameen and Sibendu Som, manager of Argonne’s Computational Multi-Physics group, collaborated with Saumil Patel, an assistant computational scientist in Argonne’s Computational Science division, who helped with pre- and post-processing, as well as developing the algorithms.
Simulation snapshot of fluid motion inside the combustion chamber of the engine. (Image by Argonne National Lab)
In the summer of 2019, with the help of Patel, Ameen was awarded computing time on Theta through DOE’s Advanced Scientific Computing Research (ASCR) Leadership Computing Challenge.
The calculations on Theta were performed with Argonne’s fluid-thermal simulation code, Nek5000, which was recognized with the Gordon Bell prize for its outstanding scalability on high-performance parallel computers in 1999.
The present day Nek5000, which scales to millions of processors, was developed primarily at Argonne. A new version, NekRS, is under development to target accelerator-based machines and has been supported in the Center for Efficient Exascale Discretizations, part of the DOE’s Exascale Computing Project.
The chief architect of the Nek5000, Paul Fischer, was consulted in the early stages of the development of the present calculations. Fischer is a senior scientist in Argonne’s Mathematics and Computer Science division and a professor in the Departments of Computer Science and Mechanical Science & Engineering at the University of Illinois at Urbana-Champaign.
After years of work to adapt Nek5000 for improved combustion modeling, the scientists performed the DNS of flow inside an internal combustion engine this spring.
“The current simulation effort is the first-ever direct numerical simulation of the flow and heat transfer inside an internal combustion engine for a real engine geometry and operating conditions,” Ameen said.
The DNS dataset generated from the current work will be useful to automotive manufacturers in several ways. The detailed information about the velocity, pressure and temperature distribution within the engine will illuminate in-cylinder processes that are inaccessible to experimentation or low-fidelity simulations. Additionally, the dataset will serve as a simulation benchmark that engine modelers can use to assess and improve the accuracy of the engineering submodels.
The research might also benefit heavy-duty engine companies.
This project was funded by the DOE’s Office of Energy Efficiency and Renewable Energy, Vehicle Technologies Office, under the Partnership on Advanced Combustion Engines consortium umbrella.
Source: Ginger Reilly
