Frontier Pushes Boundaries: 86% of Nodes Engaged on Reactor Simulation Runs

Frontier supercomputer

Details have trickled out of the Oak Ridge Leadership Computing Facility (OLCF) indicating progress in preparing Frontier, the exascale-class supercomputer ranked the world’s most powerful system, for full user operations.

Earlier this week, the Exascale Computing Project released an article on its web site entitled “Predicting the Future of Fission Power” discussing the ExaSMR (Exa for exascale; SMR for small modular reactors) toolkit for running nuclear reactor design simulations on Frontier.

Toward the end of the article, we learn that ExaSMR performed simulations on 8,192 of Frontier’s 9,472 nodes, involving more than 250 billion neutron histories per iteration, according to the article. Other than the LINPACK benchmark run last spring to certify Frontier’s exascale credentials, and Top500 ranking this is a level of system-wide engagement – drawing on roughly 86 percent of Frontier’s nodes – not previously reported.

“ExaSMR is one of several ECP applications that have had extremely positive early experiences on Frontier, including hitting their performance and capability goals,” Justin Whitt, OLCF program director, told insdeHPC. “As these science teams continue to tune their codes, we expect many more groundbreaking results over the next few months.”

The report points to progress in OLCF’s end-user readiness efforts for Frontier. There were rumors last fall that this process had run into hurdles concerning system uptimes and scaled operations, rumors that Oak Ridge dismissed as a normal part of the testing and tuning process.

The ExaSMR application integrates codes for modeling the different physics involved in reactor operations, comprising a toolkit that can predict a reactor’s fission process. The toolkit includes Shift and OpenMC for neutron particle transport and reactor depletion and NekRS, a CFD solver, for thermal fluid dynamics.

The ECP article stated that the performance achieved in these simulations is more than 100× that of the baseline simulations performed on the Titan supercomputer (the U.S.’s most powerful supercomputer in 2016) and more than double the performance improvement goal of 50× from Titan to Frontier.

Justin Whitt of the Oak Ridge Leadership Computing Facility

NekRS performed SMR simulations on up to 6,400 nodes of Frontier, including the largest reactor fluid-flow simulation performed to date with over 1 billion spatial elements. The peak performance on Frontier reflects a more than 125× improvement over corresponding baseline simulations performed on Titan, according to ECP.

“The team’s interaction with vendors and developers paid off with substantial improvements in the methods and algorithms used by the codes, yielding large gains in performance,” the article stated. “With its initial runs on Frontier, ExaSMR blew past the team’s speedup goals for its codes.”

A major focus of the ECP and its “capable exascale” strategy has been development of exascale-optimized applications. ECP states on its web site it has 24 application development teams that have worked on targeted development projects for three years.

Frontier is a Hewlett Packard Enterprise system based on the Cray EX supercomputing architecture and is powered by AMD EPYC CPUs and MI250 GPUs.