SANTA CLARA, CA, July 21, 2021 — Quobyte Inc., a developer of scale-out software-defined storage (SDS), today announced availability of its Hadoop Driver. Quobyte’s new native driver for Hadoop addresses the limitations of the Hadoop Distributed File System’s (HDFS) high-capacity design within the enterprise. The new native driver brings significant benefits in optimizing Hadoop clusters for a much wider range of applications and workloads, and true file system sharing across object storage and applications.
“Today’s analytics solutions allow enterprises to extract important insight from large volumes of data, but with the increasing prevalence of AI and machine learning in data analytics applications HDFS’s batch processing limitations have been exposed,” said Bjӧrn Kolbeck, CEO of Quobyte. “By deploying Quobyte’s native Hadoop/HDFS driver enterprises can now seamlessly share large amounts of file data with high performance across Hadoop/analytics, machine learning, and any Linux or Windows application.”
Hadoop is a powerful framework for big data processing, distributed across large amounts of commodity hardware within the enterprise. It has proven critical in helping enterprises manage the exponential growth of data, but in today’s data-driven environment, it lacks the ability to manage the real-time requirements from the new breed of analytic applications. Engineers have been working hard, with limited success, to enable small file processing by using NFS connectors.
Quobyte now offers the ability to seamlessly integrate small file processing into the Hadoop Framework, without the need for a new file system, to deliver the speedy and scalable file I/O that is needed for modern analytic and real-time applications, while integrating well with container technology.
Accelerated by the rapid evolution of analytic and real-time applications, Gartner predicts that by 2024, 50% of global storage capacity will be deployed as SDS, either on-premises or on the public cloud (up from less than 15% in 2020). With this announcement, Quobyte and Hadoop bring together the ability to run on any distributed commodity x86 hardware, that unlocks the benefits of Hadoop while removing the storage limitations of HDFS and NFS, ensuring these modern workloads can run efficiently on Hadoop clusters.
In addition, the new native driver removes security complexities by leveraging an alternative to HDFS that allows for encryption at both the storage and networking levels.
