Micron Open Source Storage Engine Designed for SSDs and Storage Class Memory

Micron announced the first open-source, heterogeneous-memory storage engine (HSE), designed specifically for solid-state drives (SSDs) and storage-class memory (SCM). 

Legacy storage engines born in the era of hard disk drives (HDDs) failed to architecturally provide for the increased performance and reduced latency of next-generation nonvolatile media. HSE, originally developed by Micron and now available to the open-source community, is ideal for developers using all-flash infrastructure who require the benefits of open-source software, including the ability to customize or enhance code for their unique use cases.

"As the only company developing storage class memory, flash and DRAM technologies, Micron is uniquely positioned to build a software stack that accelerates applications running in today's flash-based storage environments as well as storage class memory-based infrastructure of the future," said Derek Dicker, corporate vice president and general manager of the Storage Business Unit at Micron. "We have delivered a first-of-its-kind innovation for open-source storage developers, unlocking the full potential of high-performance storage applications."

In addition to delivering performance and endurance improvements, HSE reduces latency, especially for large-scale data sets, through intelligent data placement. HSE improves throughput of particular storage applications by up to six times, reduces latency 11 times1 and improves SSD endurance by seven times. HSE can also take advantage of multiple classes of media concurrently, such as flash and 3D XPoint technology. When a Micron X100 NVMe SSD, the world's fastest SSD, is added to a set of four Micron 5210 QLC SSDs, throughput more than doubles and read latency improves nearly four times.2

"We see enormous potential in the technologies being introduced by Micron, especially as it takes an innovative approach in lowering the latency between compute, memory and storage resources," said Stefanie Chiras, vice president and general manager of Red Hat Enterprise Linux at Red Hat. "We look forward to working with Micron in the open-source community to further build out these innovations, ultimately bringing new choices to the storage world derived from open standards and concepts."

"As the demand for object-based storage continues to increase and it's deployed for more and more workloads, it's no surprise that our customers are increasingly interested in fast object storage," said Brad King, field chief technology officer and co-founder of Scality. "While our storage software can support 'cheap and deep' on the lowest-cost commodity hardware for the simplest workloads, it can also exploit the performance benefits of technologies like flash, storage class memory and SSDs for very demanding workloads. Micron's HSE technology enhances our ability to continue optimizing flash performance, latency and SSD endurance without trade-offs."

Features and Benefits of the Heterogenous-Memory Storage Engine:

• Integration with MongoDB, the world's most popular NoSQL database, delivers significant performance improvements, reduces latency and takes full advantage of modern memory and storage technologies. It can also be integrated with other storage applications like NoSQL databases and object stores.
• HSE is ideal when performance at scale matters, including very large data size, large key counts (billions), high operation concurrency (thousands), or deployment of multiple classes of media.
• The platform is designed to be extensible to new interfaces and new storage devices, enabling use with a broad range of applications and solutions, including databases, internet of things (IoT), 5G, artificial intelligence (AI), high-performance computing (HPC) and object storage.
• HSE is capable of delivering additional performance for software-defined storage, such as Red Hat Ceph Storage and Scality RING, that enables cloud-native applications through containerized platforms like Red Hat OpenShift as well as tiered performance for file, block and object storage protocols for multiple use cases.
• HSE is delivered as an embeddable key-value database; Micron will maintain the code repository on GitHub.

To learn more about HSE, visit www.micron.com/hse or to start developing, visit www.github.com/hse-project.