We recently read an interesting article in Wired magazine on neuromorphic computing stating that neuroscientists increasingly regard the human brain is a “prediction machine,” that people, as a rule, are in a constant state of anticipation, extrapolation and inference. In HPC, situated as it is at the forward edge of compute power, data analysis and scientific investigation, it’s safe to say this is a pronounced trait, along with a complementary work ethic characteristic: “To strive, to seek, to find, and not to yield.”**
This predictive mindset revs up at the end of each year as the HPC community looks ahead to what may happen in the new year. Many forecasts for HPC-AI in 2023 came over our transom in recent weeks, here are excerpts from the ones we found most interesting, fresh, insightful — even contrarian.
Altair Chief Scientist Rosemary Francis:
Go Big or Go Home – HPC’s Bigger Workloads. As HPC workloads are taking on big data applications, such as in life sciences and particle accelerators like the UK’s Diamond Light Source (for greater research and experimentation), we’re seeing an explosion in workflow tools. Going into 2023, this transformation into multidimensional scheduling will be the biggest driver of change within HPC as the industry seeks to modernize itself and adapt to these big connected applications.
Dr. Rosemary Francis. Altair
HPC tackles deep learning: As deep learning becomes more prevalent in 2023, we will see a further shift in HPC workloads. While initially most machine learning workloads were run on Kubernetes or other container orchestration frameworks, it’s become clear that these systems are designed for microservices, not for the bursty, computer-intensive machine workloads now required for deep learning. Commercial HPC workload managers need comprehensive container support so organizations can spool their compute and start to take advantage of batch scheduling, cloud bursting, and fare share — all key aspects of efficient HPC.
Joe Fitzsimon, Horizaon Quantum Computing
Joe Fitzsimons, CEO of Horizon Quantum Computing, on the Death of NISQ and a Shift to Fault Tolerance in Quantum Computing
“In the last few years, applications development for quantum computing has seen a particular focus on the NISQ regime, referring to Noisy Intermediate Scale Quantum processors. The ‘noise’ in this title refers to the susceptibility of qubits to interference from environmental factors, which range from the proximity of other qubits to collisions from cosmic rays. This noisiness introduces potentially fatal errors in the processes of quantum computation. It has long been known that it is, at least theoretically, possible to build quantum computers that incorporate error correction, so that an essentially perfect computer can be built from imperfect components. However, the focus of NISQ research has been on developing variational algorithms which are hoped to be robust to minor perturbations caused by environmental noise, allowing for quantum advantage without error correction.
“Unfortunately, there is relatively little evidence that such NISQ algorithms will in fact yield an advantage over conventional computers for the broad range of optimisation and machine learning tasks for which they are being considered. While there is good reason to believe that early quantum advantage may be seen in fields such as chemistry, where the problem to be solved is quantum mechanical in nature, there are signs of renewed focus on reaching the fault-tolerance regime, in which errors are actively corrected and for which there is much stronger evidence for quantum advantage.”
John Reese, Dell
Dell Technologies’ Global CTO John Roese: A Suggested Quantum New Year’s Resolutions for CIOs
‘I will establish early skill sets to take advantage of quantum.’ Quantum computing is getting real and if you don’t have someone in your business who understands how this technology works and how it influences your business, you will miss this technology wave. Identify the team, tools and tasks you’ll devote to quantum and start experimenting. Just last month we announced the on-premises Dell Quantum Computing Solution which enables organizations across industries to begin taking advantage of accelerated compute through quantum technology otherwise not available to them today. Investing in quantum simulation and enabling your data science and AI teams to learn the new languages and capability of quantum is critical in 2023.
Gideon Mendels, CEO, Comet
Contrarian Views on ML from Gideon Mendels, CEO and co-founder of MLOps platform Comet
When Data Runs Dry: Most of the improvements seen in ML have come from training models with more and more data, but we’re getting to a point when we won’t be able to do that. Some interesting research has just come out that shows we could run out of data by 2026. If this thesis holds, we’re going to stop seeing improvements unless we can build better models on the same data set.
Environmental Impact of Generative Models: Generative models are producing extremely impressive results, but it’s not clear the impact they have on an actual business. What is clear is the carbon emission impact of training these massive models. The compute requirements are insane. So it begs the question, “Are the outcomes worth the environmental cost?”
Move Away from a Software Mindset: ML has followed the course of software development thus far, but as ML matures, this approach falls apart. No single vendor can do it all. Teams today choose the best tools available that are relevant to what they are trying to do. Vendors that tried to be everything to a team are failing. For ML to reach its potential, we need to think differently to build the right ML stack for our specific business needs.
Bias is Overhyped: Bias is a concept that gets a lot of attention– and will continue to get more with the AI Bill of Rights– it’s not something that many ML practitioners are concerned with day-to-day. Of course, they account for it, but sound ML practitioners understand the issues and know what to do to prevent bias from adversely affecting outcomes.
Jonas Kubilius
Jonas Kubilius of the Oxylabs Advisory Board on Generative AI
Jonas Kubilius, co-founder and CEO at Three Thirds and member of the Oxylabs Advisory Board, anticipates an increased evolution of Stable Diffusion, GPT-3, GitHub Copilot and other content generation techniques into profitable products used by developers and content creators in real-world applications. He added that we would see an increased interest in multi-modal models that can handle text, images, audio, and other inputs for multiple tasks.
“We will start seeing a shift from using AI for static tasks, like classification, to language-model-driven interactive workflows that help people perform their tasks more efficiently,” Kubilius said.
Peter Masson, MLCommons
Peter Mattson, president of MLCommons, on Public Datasets
“We’ll face a combination of demanding new research challenges around multi-modal and conversational AI, in addition to legal, ethical and fairness concerns with web-scraped-data in current public datasets. The industry as a whole will also need to better support not only research but also widely deployed ML applications and new regulations (e.g. through industrial-quality test sets).”
To support a “next generation of public data”, Mattson forecasts a need for strong investments in datasets for the most pressing societal and technical problems, and channel that investment through open-source-like infrastructure that enables the entire community to contribute to and review the data.
Moses Guttmann, CEO and Co-Founder of MLOps platform ClearML, on ML Trends to Watch
Moses Guttman, ClearML
Automation and the ML Skills Shortage Although we’ve seen plenty of top technology companies announce layoffs in the latter part of 2022, it’s likely none of (them) are laying off their most talented machine learning personnel. However, to fill the void … on deeply technical teams, companies will have to lean even further into automation to keep productivity up and ensure projects reach completion. We expect to also see companies that use ML technology put more systems into place to monitor and govern performance and make more data-driven decisions on how to manage ML or data science teams….
ML Talent Hoarding Is Over Layoffs of ML workers are likely among the most recent hires, as opposed to the more long-term ML staff…. Since ML and AI has become a more common technology in the last decade, many big tech companies began hiring these types of workers because they could handle the financial cost and keep them away from competitors – not necessarily because they were needed. (So) it’s not surprising to see so many ML workers being laid off… However, as the era of ML talent hoarding ends, it could usher in a new wave of innovation and opportunity for startups. With so much talent now looking for work, we will likely see many of these folks trickle out of big tech and into small and medium-sized businesses or startups.
ML Project Prioritization I see machine learning projects boiled down to two types: sellable features that leadership believes will increase sales and win against the competition, and revenue optimization projects… Sellable feature projects will likely be postponed, as they’re hard to get out quickly, and instead, the now-smaller ML teams will focus more on revenue optimization as it can drive real revenue. Performance, in this moment, is essential for all business units and ML isn’t immune to that.
Unified ML One of the factors slowing down MLOps adoption is the plethora of point solutions. It’s not to say that they don’t work, but that they might not integrate well together and leave gaps in the workflow. Because of that, I firmly believe that 2023 is the year that the industry moves towards unified, end-to-end platforms built from modules that can be used individually and also integrate seamlessly with each other (as well as integrate easily with other products). This kind of platform approach, with the flexibility of individual components, delivers the kind of agile experience today’s specialists are looking for. It’s easier than purchasing point products and patching them together; it’s faster than building your own infrastructure from scratch (when you should be using that time to build models)….
NVIDIA offered a host of predictions across a range of AI and machine learning fields:
Anima Anandkumar
Anima Anandkumar, Director of ML Research, and Bren Professor at Caltech
Digital Twins Get Physical: We will see large-scale digital twins of physical processes that are complex and multi-scale, such as weather and climate models, seismic phenomena and material properties. This will accelerate current scientific simulations as much as a million-x, and enable new scientific insights and discoveries.
Generalist AI Agents: AI agents will solve open-ended tasks with natural language instructions and large-scale reinforcement learning, while harnessing foundation models — those large AI models trained on a vast quantity of unlabeled data at scale — to enable agents that can parse any type of request and adapt to new types of questions over time.
Manuvir Das, Vice President
Manuvir Das, Vice President, Enterprise Computing
Software Advances End AI Silos: Enterprises have long had to choose between cloud computing and hybrid architectures for AI research and development — a practice that can stifle developer productivity and slow innovation.
In 2023, software will enable businesses to unify AI pipelines across all infrastructure types and deliver a single, connected experience for AI practitioners. This will allow enterprises to balance costs against strategic objectives, regardless of project size or complexity, and provide access to virtually unlimited capacity for flexible development.
Generative AI Transforms Enterprise Applications: The hype about generative AI becomes reality in 2023. That’s because the foundations for true generative AI are finally in place, with software that can transform large language models and recommender systems into production applications that go beyond images to intelligently answer questions, create content and even spark discoveries….
Kimberly Powell
Kimberly Powell, Vice President, Healthcare
Surgery 4.0: Flight simulators serve to train pilots and research new aircraft control. The same is now true for surgeons and robotic surgery device makers. Digital twins that can simulate at every scale, from the operating room environment to the medical robot and patient anatomy, are breaking new ground in personalized surgical rehearsals and designing AI-driven human and machine interactions. Long residencies won’t be the only way to produce an experienced surgeon. Many will become expert operators when they perform their first robot-assisted surgery on a real patient.
Danny Shapiro
Danny Shapiro, Vice President, Automotive
Training Autonomous Vehicles in the Metaverse: The more than 250 auto and truck makers, startups, transportation and mobility-as-a-service providers developing autonomous vehicles are tackling one of the most complex AI challenges of our time. It’s simply not possible to encounter every scenario they must be able to handle by testing on the road, so much of the industry in 2023 will turn to the virtual world to help. On-road data collection will be supplemented by virtual fleets that generate data for training and testing new features before deployment. High-fidelity simulation will run autonomous vehicles through a virtually infinite range of scenarios and environments….
Rev Lebardedian
Rev Lebardedian, Vice President, Omniverse and Simulation Technology
The Metaverse Universal Translator: Just as HTML is the standard language of the 2D web, Universal Scene Description is set to become the most powerful, extensible, open language for the 3D web. As the 3D standard for describing virtual worlds in the metaverse, USD will allow enterprises and even consumers to move between different 3D worlds using various tools, viewers and browsers in the most seamless and consistent fashion.
Ronnie Vasishta
Ronnie Vasishta, Senior Vice President, Telecoms
Cutting the Cord on AR/VR Over 5G Networks: While many businesses will move to the cloud for hardware and software development, edge design and collaboration also will grow as 5G networks become more fully deployed around the world. Automotive designers, for instance, can don augmented reality headsets and stream the same content they see over wireless networks to colleagues around the world, speeding collaborative changes and developing innovative solutions at record speeds. 5G also will lead to accelerated deployments of connected robots across industries — used for restocking store shelves, cleaning floors, delivering pizzas and picking and packing goods in factories.
Bob Pette
Bob Pette, Vice President, Professional Visualization
An Industrial Revolution via Simulation: Everything built in the physical world will first be simulated in a virtual world that obeys the laws of physics. These digital twins — including of large-scale environments, such as factories, cities and even the entire planet — and the industrial metaverse are set to become critical components of digital transformation initiatives. Examples already abound: Siemens is taking industrial automation to a new level. BMW is simulating entire factory floors to optimally plan manufacturing processes. Lockheed Martin is simulating the behavior of forest fires to anticipate where and when to deploy resources. DNEG, SONY Pictures, WPP and others are boosting productivity through globally distributed art departments that enable creators, artists and designers to iterate on scenes virtually in real time.
Rethinking of Enterprise IT Architecture: Just as many businesses scrambled to adapt their culture and technologies to meet the challenges of hybrid work, the new year will bring a re-architecting of many companies’ entire IT infrastructure. Companies will seek powerful client devices capable of tackling the ever-increasing demands of applications and complex datasets. And they’ll embrace flexibility, moving to burst to the cloud for exponential scaling. The adoption of distributed computing software platforms will enable a globally dispersed workforce to collaborate and stay productive under the most disparate working environments.
Similarly, complex AI model development and training will require powerful compute infrastructure in the data center and the desktop. Businesses will look at curated AI software stacks for different industrial use cases to make it easy for them to bring AI into their workflows and deliver higher quality products and services to customers faster.
Azita Martin
Azita Martin, Vice President, AI for Retail and Consumer Products Group
AI to Optimize Supply Chains: Even the most sophisticated retailers and e-commerce companies had trouble the past two years balancing supply with demand. Consumers embraced home shopping during the pandemic and then flocked back into brick-and-mortar stores after lockdowns were lifted. After inflation hit, they changed their buying habits once again, giving supply chain managers fits. AI will enable more frequent and more accurate forecasting, ensuring the right product is at the right store at the right time. Also, retailers will embrace route optimization software and simulation technology to provide a more holistic view of opportunities and pitfalls.
Malcolm deMayo
Malcolm deMayo, Vice President, Financial Services
Cloud-First for Financial Services: Banks have a new imperative: get agile fast. Facing increasing competition from non-traditional financial institutions, changing customer expectations rising from their experiences in other industries and saddled with legacy infrastructure, banks and other institutions will embrace a cloud-first AI approach. But as a highly regulated industry that requires operational resiliency, an industry term that means your systems can absorb and survive shocks (like a pandemic), banks will look for open, portable, hardened, hybrid solutions. As a result, banks are obligated to purchase support agreements when available.
David Reber
David Reber, Chief Security Officer
Data Scientists Are Your New Cyber Asset: Traditional cyber professionals can no longer effectively defend against the most sophisticated threats because the speed and complexity of attacks and defense have effectively exceeded human capacities. Data scientists and other human analysts will use AI to look at all of the data objectively and discover threats. Breaches are going to happen, so data science techniques using AI and humans will help find the needle in the haystack and respond quickly.
Kari Briski
Kari Briski, Vice President, AI and HPC Software
Unlabeled Data Finds Its Purpose: Large language models and structured data will also extend to the reams of photos, audio recordings, tweets and more to find hidden patterns and clues to support healthcare breakthroughs, advancements in science, better customer engagements and even major advances in self-driving transportation. In 2023, adding all this unstructured data to the mix will help develop neural networks that can, for instance, generate synthetic profiles to mimic the health records they’ve learned from. This type of unsupervised machine learning is set to become as important as supervised machine learning.
The New Call Center: Keep an eye on the call center in 2023, where adoption of more and more easily implemented speech AI workflows will provide business flexibility at every step of the customer interaction pipeline — from modifying model architectures to fine-tuning models on proprietary data and customizing pipelines. As the accessibility of speech AI workflows broadens, we’ll see a widening of enterprise adoption and giant increase in call center productivity by speeding time to resolution. AI will help agents pull the right information out of a massive knowledge base at the right time, minimizing wait times for customers.
Deepu Talla
Deepu Talla, Vice President, Embedded and Edge Computing
Robots Get a Million Lives: More robots will be trained in virtual worlds as photorealistic rendering and accurate physics modeling combine with the ability to simulate in parallel millions of instances of a robot on GPUs in the cloud. Generative AI techniques will make it easier to create highly realistic 3D simulation scenarios and further accelerate the adoption of simulation and synthetic data for developing more capable robots.
Marc Spieler
Marc Spieler, Senior Director, Energy
AI-Powered Energy Grid: As the grid becomes more complex due to the unprecedented rate of distributed energy resources being added, electric utility companies will require edge AI to improve operational efficiency, enhance functional safety, increase accuracy of load and demand forecasting, and accelerate the connection time of renewable energy, like solar and wind. AI at the edge will increase grid resiliency, while reducing energy waste and cost.
