Danny Perez from LANL
In this edition of Let’s Talk Exascale, Danny Perez of Los Alamos National Laboratory describes the EXAALT program for molecular dynamics.
As part of the Exascale Computing Project, EXAALT has produced and released an open source software package that integrates three large pieces of code. The integrated code is designed to allow for molecular dynamics simulations with longer timescales, huge systems of atoms and molecules, and high-accuracy semi-empirical quantum capability.”
Researchers can run computer simulations of the physical movements of atoms and molecules and make inferences about the dynamic evolution of the system. This method of simulation, called molecular dynamics, is used at many computing centers across the country in areas such as materials science and biology. These simulations can yield extremely detailed understanding of the mechanisms by which materials evolve in time and in response to external stimuli. However, exascale computing will require a comprehensive molecular dynamics capability with greater versatility.
The coming exascale computing systems will create the necessity for new molecular dynamics codes to take advantage of the leap in power and performance. In fact, if today’s molecular dynamics codes were run on an exascale machine, larger numbers of atoms and molecules could be simulated, but longer times could not.
The reason for the limitation is that conventional algorithms exploit large computers by decomposing space into small cells and putting individual processors in charge of each one. This approach works well if cells are large, but if they become too small because atoms are spread thin across compute resources in an effort to further increase the simulation speed, the overhead of synchronizing the work over different cells begins to dominate, and performance plummets. This impediment has, for many years, confined improvement in simulation times.
Overcoming the Limitations
The EXAALT (Exascale Atomistic Capability for Accuracy, Length, and Time) project is endeavoring to push past the current limitations and allow for simulations with not only longer length scales but also longer time scales and higher accuracy.
Computationally, EXAALT’s goal is to develop a comprehensive molecular dynamics capability for exascale.
The user should be able to say, ‘I’m interested in this kind of system size, timescale, and accuracy,’ and directly access the regime without being constrained by the usual scaling paths of current codes,” said Danny Perez of Los Alamos National Laboratory (LANL) and the EXAALT team.
Simulation of helium on nuclear fusion reactor wall
When simulating the evolution of materials, accessing very long times can be crucial. For example, in the case of the growth of helium bubbles in the walls of nuclear fusion reactors, imposing fast growth rates leads to drastically different predictions than when using EXAALT to extend the simulation to timescales that are closer to realistic conditions. Courtesy: the EXAALT project
Users need such a capability to understand materials for nuclear energy, both nuclear fuels in fission power plants and on the walls of fusion reactors.
We aim to build a comprehensive capability and demonstrate it on nuclear applications, but, really, it’s a very general framework that anybody else in materials science should be able to use,” Perez said.
One of EXAALT’s main targets is to allow for the development of better materials, because the national need is so great. For example, hundreds of millions of tons of metal are consumed in the United States each year. However, the development process for a new material takes a long time and is error prone.
We hope that exascale will give us the ability to run simulations directly in the conditions that are relevant to the applications,” Perez said. “This will really help in terms of the design and testing of novel materials, which is important in scientific discovery, but also for industrial research. And since we focus on materials in extreme conditions, our work has impact on the national security side of DOE’s mission as well.”
Providing a Versatile Product
The EXAALT project has produced and released an open source software package that integrates three large pieces of code developed at LANL and Sandia National Laboratories (SNL): ParSplice, an accelerated molecular dynamics module; LAMMPS, a well-known molecular dynamics code; and LATTE, a LANL-produced code.
The integrated code is designed to allow for molecular dynamics simulations with longer timescales (via ParSplice), huge systems of atoms and molecules (via LAMMPS), and high-accuracy semi-empirical quantum capability (via LATTE), to make approximations and obtain some parameters from empirical data. In time, users will be able to dial in the regime they are interested in, set up their system, and then launch EXAALT on a large machine.
EXAALT has made tremendous progress in the last year,” Perez said. “A focus has been on the development of methods that can simulate intermediate-size systems for long times. This regime is very relevant to many applications in materials science, such as the evolution of the walls of fusion reactors.”
A Solution for Intermediate-Size Systems
Perez explained that simulating intermediate-size systems is difficult. He said the reason is that it requires systems that are too small to fully utilize an exascale machine with traditional molecular dynamics tools, yet too large for conventional accelerated molecular dynamics methods. The waiting times between morphological changes anywhere in the system, he said, become so short that the simulation cannot be further accelerated.
The EXAALT team has implemented a generalization of the ParSplice method that allows for different sections of the systems to be accelerated separately in short bursts before being synchronized back together,” Perez said. “In this case, the efficiency of ParSplice becomes controlled by the timescale over which morphological changes occur locally in each section and not by the much shorter global timescale. This allows for much better performance.”
To demonstrate the scalability of this approach to the application of accelerated dynamics methods, the team has run at scale using 270,000 cores on the Theta supercomputer at the Argonne Leadership Computing Facility. This simulation also employed a new generation of materials model that the team is developing. In addition, the EXAALT team demonstrated quantum simulations of nuclear fuels at scale—again using 270,000 cores on Theta—by employing a combination of ParSplice, LAMMPS, and LATTE.
Near-Term Plans
A key next step is to ensure that EXAALT can make the most of the latest computer architectures that rely heavily on accelerators, such as GPUs, to deliver very high simulation rates. This requires the careful redesign and optimization of key components of EXAALT. This essential effort is currently ongoing in collaboration with different projects within ECP and with the National Energy Research Scientific Computing Center.
Source: Exascale Computing Project
