A new supercomputer has been deployed at the Jülich Supercomputing Center (JSC) in Germany. Called QPACE3, the new 447 Teraflop machine is named for “QCD Parallel Computing on the Cell.”
QPACE3 is being used by the University of Regensburg for a joint research project with the University of Wuppertal and the Jülich Supercomputing Center for numerical simulations of quantum chromodynamics (QCD), which is one of the fundamental theories of elementary particle physics. Such simulations serve, among other things, to understand the state of the universe shortly after the Big Bang, for which a very high computing power is required.
The demand for high performance computers to solve complex applications has risen exponentially, but unfortunately so has their consumption of power. Many supercomputers require more than a megawatt of electricity to operate and annual electricity costs can easily run into millions of Euros. The energy supply is therefore a significant part of the operating costs of a data center. According to recent analyst studies, this represents the second-largest factor in addition to personnel and maintenance costs. The upcoming boom with (3D) video streaming, augmented reality, image recognition and artificial intelligence is driving up the demand for data center capabilities, thereby placing new challenges in the power supply sector.
So for exactly this reason, the goal of the project QPACE was to become an energy-efficient supercomputer and why scientists at the University of Regensburg, Faculty of Physics, headed by Prof. Dr. Tilo Wettig, have been working with Fujitsu to achieve as much computing power as possible, with the minimal energy consumption. Together with the QPACE project team, Fujitsu has built an HPC cluster based on the FUJTSU Server PRIMERGY CX600 M1.
The PRIMERGY CX600 M1 features a completely new server architecture purpose-built to meet the exacting demand for massive parallel computing power from today’s high performance computing applications in the fields such as scientific research, product development and business intelligence. The system comprises the PRIMERGY CX600 M1 chassis and PRIMERGY CX1640 M1 server nodes, which deliver ultra-dense clusters and a highly competitive price/performance ratio.
The entire QPACE3 HPC Cluster consists of 44 PRIMERGY CX600 M1 multi-node chassis, which are equipped with a total of 352 PRIMERGY CX1640 M1 server nodes. Each of these nodes are equipped with an Intel Xeon Phi processor with 64 cores, making it ideal for HPC workloads that require high thread parallelism, large vectors, and significant memory bandwidth. Moreover the QPACE system makes use of a high-performance communication architecture based on Intel Omni-Path technology. This communication architecture offers low communication latency, low power consumption and a high throughput.
In order to further optimize the electricity costs in the data center, hot and cold air isolation is often used. Fujitsu further improves this concept with the award-winning Cool-Central Liquid Cooling Solution that is also used in the QPACE3 project. Liquid Cooling is a “free cooling” solution (by external air) that captures between 60% and 80% of server heat. The solution removes heat from selected components (e.g. CPUs) within servers using hot water cooling with a flow temperature of 40⁰C. This helps to saves significant energy costs.
On request of the customer, an initial configuration of the system was set up and put into operation at the Fujitsu Datacenter in Augsburg in October 2016. In this first configuration level, 288 compute nodes with a total of 18,432 cores have been installed in three PRIMECENTER LC racks, connected to the existing water cooling system and measured under full load using the LINPACK benchmark.
In this benchmark, the cluster achieved a measured computing power of 447 TFLOPS (Tera Floating-Point Operations Per Second) with a measured power consumption of 77 kW (of which about 50 kW are water-cooled). With the resultant ratio of computing power to power consumption (5.8 GFLOPS / W), the QPACE3 system reached rank 5 in the current GREEN500 list (as of November 14th, 2016). This list shows the 500 most powerful supercomputer systems in the world sorted by energy efficiency (performance / watt).
Sign up for our insideHPC Newsletter
