Sponsored Post
This technology guide, “insideHPC Guide to Composable Disaggregated Infrastructure (CDI) Clusters,” will show how the Silicon Mechanics Miranda CDI Cluster™ reference architecture can be a CDI solution blueprint, ideal for tailoring to specific enterprise or other organizational needs and technical issues.
The guide is for a technical person, especially those who might be a sys admin in financial services, life sciences or a similarly compute-intensive field. This individual may be tasked with making CDI work with a realizable ROI or just finding a way to extend the value of their IT investment while still meeting their computing needs.
Introduction
IT infrastructure carries out a critical role in an organization’s success, and ultimately its survival, by empowering an organization to achieve its objectives in the time frame required to provide what the market demands. To achieve this advantage, enterprise stakeholders must solve the problem of managing infrastructure capacity as well as compute cycles.
IT organizations have a fixed amount of resources and must maximize their utilization to deliver the compute environment their organization needs when needed. However, today, once these resources are no longer required, they cannot be redeployed to another application with the same level of precision. Composable disaggregated infrastructure (CDI) is the key to solving this optimization problem. It enables valuable resources to be deployed through software in just the right levels and within a minimal amount of time, even
inside an HPC or AI cluster.
Legacy data center infrastructures were not designed for today’s HPC and AI workflow requirements. Their requirements can fluctuate significantly from one application to the next and even vary within the same application. These changing application requirements make it problematic for traditional infrastructure to
meet the demands of advanced workloads.
As a result, many IT departments—especially those in enterprises—are now looking toward CDI.
A scalable modern data center that supports HPC or AI needs an infrastructure solution that’s able to integrate compute, storage, and communication I/O into a single-system cluster fabric, scaling resources up and out across the cluster as required. This solution frees resources from their silos so they can be shared with other network users who draw from a series of resource pools through CDI.
CDI is an efficient way to increase resource utilization and assign resources to an application from disaggregated pools of compute, storage, and networking—along with additional resources providing GPU, FPGA, and storage memory accelerators—that can be provisioned on an on-demand basis.
Provisioning is implemented with a programmable API that manages all the IT resources through a central application. This software-defined management layer is used to discover, allocate, and manage the pools of disaggregated resources, ensuring that the correct assets are in the right place at the right time. CDI offers rapid deployment across multiple fabrics without vendor-lock, while optimizing resource utilization and keeping pace with future business needs.
Latency is a well-known confounder of modern HPC & AI performance which depresses utilization rates and reduces TCO. CDI is an emergent software defined infrastructure solution that targets the data center to maximize IT resource usage and improve business agility by reducing latency, while advancing TCO in several other ways as well.
CDI also serves as an alternative to the cloud. At its core, CDI is a set of disaggregated resources connected by a PCIe-based fabric that enables you to dynamically provision bare metal instances. It uses cloud native design principles to deliver best-in-class flexibility and performance. This orientation provides the flexibility of the cloud and the value of virtualization but the performance of bare metal. CDI offers the ability to run diverse workloads on a cluster while still optimizing for each workload.
Benefits of CDI Clusters to the Enterprise
It’s important to review design best practices for CDI with respect to HPC and AI. Enterprises will benefit from a flexible infrastructure solution in terms of the overarching value of CDI, the flexibility of the underlying hardware, different workloads, environments, as well as benefits like cost effectiveness, and scalability compared to cloud services, and cloud service providers.
With CDI, there is extraordinary, high-speed networking between the nodes of a cluster. Further, CDI enables dynamic, bare-metal composability utilizing pools of off-the-shelf hardware connected across intelligently managed fabrics. With the ability to compose across PCIe, Ethernet, and Infinband, CDI dramatically increases the utilization, flexibility and effective use of valuable data center assets. When an organization embraces CDI, they realize, on average, a 2x to 4x increase in data center resource utilization.
Over the next few weeks we will explore these topics:
- Introduction, Benefits of CDI Clusters to the Enterprise
- Technology Use Case Examples, CDI and the Enterprise Infrastructure of the Future
- Key Ingredients of a High-performance CDI Cluster, Conclusion
Download the complete insideHPC Guide to Composable Disaggregated Infrastructure (CDI) Clusters courtesy of Silicon Mechanics.
