In this special guest feature from the SC19 Blog, Dan Jacobson and Wayne Joubert from ORNL describes how the Summit supercomputer is helping untangle how genetic variants, gleaned from vast datasets, can impact whether an individual is susceptible (or not) to disease, including chronic pain and opioid addiction.
Researchers are using high performance computing to tackle the opioid crisis. The team’s statistical and computational procedures resulted in the fastest-ever scientific calculation—2.36 exaflops on 99 percent of Summit. It also earned them a share of the 2018 Gordon Bell Prize, awarded for outstanding achievement in HPC that emphasizes innovation in applications to science, engineering, and large-scale data analytics..
Charity Plata: What led you to using Summit’s compute power toward something like examining root causes of the opioid crisis, which seems as far away from HPC computing as you can get?
Dan Jacobson from ORNL
Dan Jacobson and Wayne Joubert: Biological systems are incredibly complex and the possible combinatorial interactions among the components of a human cell are on the order of 10170. Complex traits and behaviors, such as addiction, will involve the interactions of many components and, as such, are very challenging HPC problems. Thus, to better understand biological systems, the use of HPC is actually central to our efforts.
Charity Plata: It appears that many ideas for examining genetic architectures at the population scale have existed for some time. Thus, in addition to its sheer size, what else about the Summit system (and HPC) made this work viable right now?
Dan Jacobson and Wayne Joubert: Traditional approaches to genetics, be it Mendelian or traditional genome wide association studies, known as GWAS, make the assumption that genes/alleles operate independently. Organisms are highly interconnected systems, so this is a false premise for many traits. Epistasis, the idea that alleles (mutations) interact as sets to effect traits, is increasingly being shown to be the rule rather than the exception. However, testing for epistasis takes us to these huge combinatorial problems that have been computationally infeasible before—and, as such, a bit of an inconvenient truth in biology. The combination of new algorithm design with the power of Summit is finally allowing us to solve problems like this that we have been dreaming of for a long time. The tensor cores on Summit’s GPUs allowed us to unleash unprecedented amounts of compute power. The large memory footprint of each node allowed us to tackle a size of problems that we could not previously, the NVLink connection between CPU and GPU along with adaptive routing between nodes allowed use to parallelize and scale these problems incredibly efficiently, and the use of the burst buffers allowed us to minimize the impact of I/O (input/output).
Charity Plata: How was this genomics research a good way to assess Summit’s effectiveness/efficiency? And, to use car parlance, were you able to engage Summit with a “wide open throttle?”
Dan Jacobson and Wayne Joubert: The genomic problem provided a complex and challenging problem that required us to use the entirety of Summit for long runs. As such, we were indeed running Summit with a wide open throttle. The power-traces from these runs indicate that this application runs hotter than other applications on Summit and thus is really maximizing use of the available compute power.
Charity Plata: Your team’s work builds a distinct bridge between biology and computer science. In what ways do you expect domain and computer scientists’ work to evolve together when seeking to resolve grand challenge problems in applications beyond biology (e.g., environmental science or chemistry)?
Dan Jacobson and Wayne Joubert: We also work in areas of environmental science and the effects of chemistry on the environment. In fact, some of our most recent work has been targeted at understanding environments at a planetary scale. As such, I think we can attest to the continued power of the tight integration between domain and computational scientists.
Charity Plata: What innovations in both HPC and genomics were realized as a result of this work? Did anything surprise you along the way?
Wayne Joubert is a Computational Scientist at ORNL.
Dan Jacobson and Wayne Joubert: Ours was the first project to use tensor cores to break the exascale barrier and is still the fastest calculation ever achieved. The genomics results form a foundation for many of our ongoing studies so are part of many manuscripts now and in the future. In short, this is allowing us to make new scientific discoveries that we simply could not have done before. There are ongoing applications of this work in a broad range of fields, including, but not limited to, precision medicine, agriculture, bioenergy, and ecosystems research. One of the surprises has been unexpected patterns and biological signatures that these approaches are revealing, well beyond the expected results.
Charity Plata: What did it mean to your team to earn a share of the 2018 Gordon Bell Prize, and how does it further validate your research?
Dan Jacobson and Wayne Joubert: We are most excited about all of the new science that this work enables. I am particularly proud that we were the first group to break the exascale barrier and that we did it for biology, thus demonstrating the importance of supercomputing for complex biological problems. Receiving the Gordon Bell has been a nice validation of this high-end convergence between biology and supercomputing.
Charity Plata: Can you share any helpful hints for anyone who may be considering if her/his research merits a Gordon Bell Award submission?
Dan Jacobson and Wayne Joubert: Pick an incredibly challenging problem, map it to the right collection of compute architectures, start as far ahead of time as you can, plan for a lot of failures along the way, and work very, very hard.
Charity Plata: What does SC19’s theme, ‘HPC is Now’, mean to you?
Dan Jacobson and Wayne Joubert: The importance of HPC in an increasingly data-driven world is very apparent right now, not just sometime in the future.
Registration is now open for SC19, which takes place Nov. 17-22 in Denver.
