In this sponsored post, Ian Lumb, Technical Writer at Sylabs, shares the origins of the new annual Singularity User Group Meeting, a result of teamwork between Sylabs and The San Diego Supercomputer Center (SDSC).
Ian Lumb, Technical Writer, Sylabs
The key takeaway here is plain and simple: The inaugural meeting of the Singularity User Group (SUG) will be hosted in San Diego, March 12-13, 2019. How Singularity got to this point, however, is a story worth sharing.
In 2015, HPC users at the Lawrence Berkeley National Laboratory expressed their interest in containerizing applications and workflows to then HPC Systems Architect and Technical Lead Gregory Kurtzer. At this point, it was mostly just an interest in making use of the paradigm, as existing solutions for containerization simply were not a very good fit for these HPC workloads.
Kurtzer, who had previously founded CentOS Linux, Warewulf, and other open source projects, listened to the Lab’s users, and teased out the requirements necessary to develop Singularity containers for HPC, and compute-driven workloads in general. Roughly three-years later, Kurtzer’s Sylabs Inc. officially emerged from stealth mode, aiming to provide the market with solutions and services based upon Singularity containers.
Probably the most significant containerization differentiator is that Singularity emphasizes integration over isolation. This means that accessing HPC-typical devices such as GPUs and InfiniBand interconnects is easily achieved with close to bare-metal performance and latency characteristics — claims that have been recently validated through benchmarking studies from the Argonne Leadership Computing Facility (ALCF).
The Singularity Image Format (SIF) encapsulates the entire container runtime into a single, immutable, plus cryptographically signable and verifiable file. Thus this additional differentiator allows untrusted users to execute untrusted containers in a trusted way. SIF’s ability to encapsulate any Open Containers Initiative (OCI) compatible format (e.g., Docker images) into a native runtime for Singularity has served to accelerate community adoption — a community that today numbers some 40,000 users who run many millions of SIF-containerized applications per day.
Within this community, the San Diego Supercomputer Center (SDSC) was amongst the first sites to adopt Singularity. Given that it has a history of supporting and promoting the software, SDSC and Sylabs together decided that the timing was right to offer the Singularity user, developer, and provider community a compelling event. Thus the inaugural meeting of the Singularity User Group (SUG) will be hosted in beautiful San Diego at SDSC, March 12-13, 2019. Structured to be accessible to those who are brand new to containerization, a hands-on session will allow for a rapid onramp to Singularity. Then, for the remaining day-and-a-half, presentations that include demos will allow for technically deeper dives from all stakeholders in the community. Keynotes from Kurtzer himself, as well as leadership use cases from SDSC, are certain to draw attention.
Accepted contributions to cover high throughput computing, system telemetry and I/O profiling, reproducible science, best practices for containerizing InfiniBand, national software repositories for interactive HPC, and more!
Although a tentative agenda has been struck to provide a glimpse into the technical program, it is by no means complete. Anchored by the keynotes alluded to above, and contributions from members of the Sylabs engineering team, the big reveal is yet to come — as a work in progress, it is those talks contributed by the Singularity user, developer, and provider community that will take this event to the next level. The reason we can share this with certainty is that we have already received a number of compelling submissions in response to our Call for Papers/Presentations. By the time you read this, this CFP will be nearing its close; however, we still encourage you to make a submission as we intend to be as inclusive as possible.
Singularity has become what it is today through the engagement of users, developers, and providers that have collectively developed a sense of community around the software and its ecosystem. The success of SUG similarly hinges upon community engagement. Even if you are not quite ready to share your experiences through a talk at SUG, we hope you will join us by registering for, and then participating in person at, the event. Not only will you have the pulse of this extremely important software innovation for compute-driven workloads, you will leave with insight into where it is heading.
We look forward to meeting you in San Diego!
Ian Lumb is a Technical Writer at Sylabs.
