Lenovo & Intel power this reconstruction of the visual system of a tiny wasp, which required 500 GB of RAM. Credits: Flatiron Institute/CCB, Janelia Farm/HHMI, Micro Insect Lab/Moscow State University.
Today Lenovo and Intel revealed how their collaboration on joint technology solutions that accelerate the convergence of HPC and AI are helping scientists at the Flatiron Institute solve scientific challenges in entirely new ways.
The Flatiron Institute, located in New York City, is an internal research division of the Simons Foundation, whose mission is to advance the frontiers of research in mathematics and the basic sciences. Its Scientific Computing Core supports scientific disciplines that require an HPC infrastructure to handle a variety of extremely large data files, from giant genomic sequencing files to 100,000 small files in a single directory. Flatiron’s on-site cluster – a central HPC infrastructure – is comprised of 17,000 Intel computing cores in Lenovo ThinkSystem SD530 servers, running HPC and AI workloads. This combination allows the computers to analyze large, rich datasets and simulate complex physical processes.
We see high-performance computing and artificial intelligence as the keys to unlocking an entirely new era of untapped potential for customers,” said Scott Tease, general manager for HPC and AI at Lenovo’s Data Center Group. “Seeing some of the greatest minds rely upon our technology to discover answers for some of humanity’s most pressing questions is rewarding. But pushing the boundaries of HPC and AI capabilities even further so more people can benefit is what drives us.”
Scientific Progress in an Exascale Era
Researchers at the Flatiron Institute are pursuing advances in biological sciences, astrophysics, quantum physics, and computational mathematics. Yet, before the large-scale computing clusters, scientists at the Flatiron Institute were limited in their scientific progress by system constraints that imposed practical limitations in the speed and scope of their system-based research. Lenovo and Intel partnered with the Flatiron Institute to dramatically improve and expand its HPC and AI capabilities so technologies are no longer the limiting factor in its research.
Cosmological simulations like galaxy formation and black hole creation require hundreds of thousands of cores connected to each other to run. Science should dominate our researchers’ time, not computation,” said Dr. Ian Fisk, Scientific Computing Core co-director at the Flatiron Institute. “Our goal is that the limiting factor in research should be the carbon-based systems rather than the silicon-based systems.”
Solutions Powered by HPC and AI Convergence
Research at the Flatiron Institute is conducted with the help of Lenovo ThinkSystem SD530 Dense Rack servers with Lenovo Neptune Thermal Transfer Module (TTM) technology running leading edge Intel Xeon Platinum 8268 processors, part of the 2nd Gen Intel Xeon Scalable family featuring Intel Deep Learning Boost technology. Leveraging the performance and reliability of the joint Lenovo-Intel solution, researchers at the Flatiron Institute today are not only able to process petabytes of data faster and more efficiently to accelerate the pace of research. They can now carry forward workload-intensive research projects that otherwise would simply not have taken place.
For example, in the past decade, the cost of full genome sequencing has become more affordable as HPC has become more attainable. Scientists previously could only sequence about 2% of genomic data, but they can now look at the entire genomic sequence of thousands of families at once. This progress can be the key to more effective discovery of genes that cause disease or the development of precision medicine.
Due to their unique computing demands and limited space constraints, Flatiron needed a dense, air-cooled system with vast amounts of memory. “With our Lenovo HPC supercomputer cluster, we have four or five times more memory than a typical HPC farm,” Fisk said referring to the additional RAM available with each node. “Reading a 200 GB genomic sequencing file for a biology project or making 11,000 connections to other nodes for an astrophysics project requires a lot of memory. Now, when a researcher comes along with an interesting challenge that their equipment doesn’t support, we can achieve that.”
Dr. Fisk said the Lenovo – Intel systems were the fastest and densest he had seen, praising them as “a workhorse,” and noting the importance of the system’s improved thermals so those intense workloads can operate in a small space.
Breakthrough research, like the amazing work being done at the Flatiron Institute, requires breakthrough technology,” said Trish Damkroger, vice president and general manager of the Extreme Computing Organization at Intel. “Through our close partnership with Lenovo, we are able to deliver these leading technologies, including the only CPU with both AI and HPC acceleration built in, to help Flatiron work to solve some of the world’s most complex issues with faster time to insights.”
Lenovo and Intel have extended their long-standing collaboration in the data center with a multiyear commitment to accelerate the rapidly growing convergence of HPC and AI solutions to organizations of all sizes. The extended partnership focuses on three key areas: systems and solutions, software optimization for HPC and AI convergence, and ecosystem enablement.
