By Bill Wagner, CEO Bright Computing
Bright Computing has been elaborating on the theme of “Managing Complexity in the New Era of HPC” for a number of months, receiving coverage in InsideHPC, and delivering a recent webinar on that very subject as part of the SC20 Exhibitor Forum.
Now, the state of HPC isn’t so much changing as much as it is expanding. All the traditional HPC modeling and simulation applications, use cases, and technologies that you know today will continue for many years to come. However, there is a trend in the industry that is bringing with it a diversification of these use cases and technologies. The HPC workloads of today will (and in a lot of cases already are) be joined by machine learning workloads, like training deep learning algorithms, and with data analytics workloads. Going forward, we will see new processors and accelerators from various companies within the industry, finding their way into more HPC systems. In addition to bare-metal systems that dominate HPC systems today, companies like VMware have closed the performance gap in HPC with virtual machines that now provide suitable performance for many HPC applications. The popularity and value of Kubernetes is showing that HPC containers will be a reality in the years ahead. And finally, we have cloud and edge HPC applications. HPC in the cloud is one of the fastest-growing segments of HPC spending today. And with the diversification of use cases in edge computing, it is not a stretch to envision edge applications becoming sophisticated enough or abundant enough to require clusters at the edge in the near future.
Let’s take a minute to consider one of the broadening commercial use cases in HPC―machine learning. While the promise of machine learning is pulling more and more organizations into the realm of HPC, many of these organizations have little to no experience building and managing these systems. Even the most experienced of HPC practitioners will tell you that building and managing HPC Linux clusters can be challenging, and the addition of machine learning applications in HPC environments has the potential to increase that complexity exponentially. With thousands of hardware and software elements that must work in unison, spanning compute, networking, storage, etc., coupled with the skills and knowledge required to put it all together, the combination can easily become more than an organization can cope with.
In this Stat Sheet produced by Intersect360, they found that 70% of HPC users are running machine learning environments or are working to implement one within the next year; another 20% are considering it. They also found that more than 50% of the HPC users that are incorporating machine learning have seen a budget increase of at least 5% as a direct result; some budgets have more than doubled.
These trends will continue as HPC propagates and infiltrates industries and organizations that had not previously worked with HPC. AI, for example, continues to have a positive impact in a variety of industries with use cases such as:
- Autonomous driving
- Real-time fraud detection
- Computer vision monitoring manufacturing lines, shaping consumer retail experiences, and improving medical diagnosis
- Sensor-driven AI optimizing agricultural crop conditions and yields
- Accelerated drug discovery
- Business intelligence apps built on machine learning
The fundamentally different applications and technologies associated with these example use cases are changing the landscape of HPC systems, necessitating new capabilities, and a new level of flexibility like never before―and the implication of this for HPC systems is increased complexity. Traditional HPC environments are notoriously complex, and the addition of new machine learning and analytics applications, integration with public cloud and edge, support for Kubernetes and new processors/accelerators and more, increases that complexity dramatically. For organizations facing this “new era of HPC”, there is an urgency to get business value from their investment and the necessity for an “easy button” to help conquer all this complexity.
This brings us to Bright Computing. Bright just launched version 9.1 of Bright Cluster Manager software at SC 20. Bright Cluster Manager automates the process of building and managing high-performance Linux clusters from edge-to-core-to-cloud, eliminating complexity, and making clusters flexible, adaptable, and accelerating an organization’s time to value.
Bright has a rich history of helping HPC system administrators and end-users leverage new and emerging technologies. The University of North Dakota’s (UND) Computational Research Center is one example of an HPC end-user that leveraged Bright to unify their newly designed high-performance infrastructure and give them more versatility and access to machine learning resources. Aaron Bergstrom, Advanced Cyberinfrastructure Manager at UND, noted, “Bright software makes it easy for us to train new staff to deploy, provision, and manage our clustered infrastructure for HPC, Data Science, and Cloud; all integrated, and all from a single point of control. We can deploy Ceph nodes, install standard HPC products, roll out cloud features, introduce data science packages, all from a single interface with robust enterprise-grade support.”
UND chose Bright Cluster Manager for its integrated approach for building and managing HPC clusters that reduces complexity, accelerates time to value, and provides enormous flexibility. Bright Cluster Manager now provides UND with automated cluster provisioning and setup, including pre-configured machine learning frameworks and libraries that accelerate data science projects and supports UND’s mission to add data science to its research agenda. With Bright Cluster Manager, UND can now easily extend to the public cloud for additional resources, re-image cluster nodes on the fly to host new applications, and so much more. UND has access to a robust framework that automates the process of building and managing a high-performance infrastructure that delivers Clusters-as-a-Service for machine learning, data analytics, HPC, and more.
Intersect360 further states that Bright Cluster Manager is the most-cited commercial system management package among HPC users; we provide the technology that organizations are turning to in order to embrace the new era of HPC and manage the associated complexity. The report puts it succinctly; “Fundamentally, Bright Computing helps address the big question in HPC: how to match diverse resources to diverse workloads in a way that is both efficient today and future-proof for tomorrow.”
As more organizations take the leap into HPC, Bright Computing aims to be the company that helps solve the challenge of complexity within the industry and replace it with flexibility, ease of use, and accelerated time to value.
To learn more about our AI and machine learning resources, please click here to watch our recent webinar. You can also visit our website or email us at info@brightcomputing.com for more information or to speak to one of our solution specialists.
