Co-Designing NVM-based Systems - for Machine Learning and In-memory Search Applications

Aug 2, 2026

This invited paper examines how non-volatile memory (NVM) technologies can improve machine learning workloads on resource-constrained edge devices and accelerate large-scale search. It analyzes energy efficiency and latency reduction via in-memory computation and different NVM write modes, while addressing PCM write-latency, endurance, and retention trade-offs and the read-destructive wear implications of FeRAM. It also evaluates NVM-based CAM/CIM accelerator architectures, focusing on how device choice and array partitioning/merge schemes affect performance, area, latency, energy, and accuracy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Abstract

I. Introduction

Edge ML with NVMs: PCM and FeRAM trade-offs

NVM-based in-memory search: CAM/CIM architecture choices

Weight-access challenges in deep learning and limitations of eDRAM

Processing-in-NVM concepts and analog-domain constraints

(Invited Paper) Co-Designing NVM-based Systems for Machine Learning and In-memory Search Applications Jrg Henkel∗ , Lokesh Siddhu∗ , Hassan Nassar∗ , Lars Bauer, Jian-Jia Chen†, Christian Hakert†, Tristan Seidl†, Kuan Hsun Chen‡, Xiaobo Sharon Hu§ , Mengyuan Li§ , Chia-Lin Yang¶ , Ming-Liang Wei¶ ∗ Karlsruhe Institute of Technology, Germany, †TU Dortmund, Germany, ‡University of Twente, the Netherlands, § University of Notre Dame, Indiana, USA, ¶ National Taiwan University, Taiwan Abstract—With the rapid development of the Internet of Things, machine learning applications on edge devices with limited resources face challenges due to large data scales and irregular memory access patterns. Non-volatile memory (NVM) technologies provide promising solutions by offering larger capacity, low leakage power, and data persistence. In this paper, we discuss the potential of NVM technology in enhancing machine learning applications by improving energy efficiency and reducing latency through in-memory computation and different NVM write modes. The insights from this analysis provide valuable guidance to device researchers and system architects working to develop highperformance systems for machine learning and accelerators in large-scale search applications using NVMs. I. INTRODUCTION In the rapidly evolving field of machine learning (ML), especially on edge devices with constrained resources, the integration of Non-Volatile Memories (NVMs) such as Phase Change Memory (PCM) offers notable advantages. PCM’s larger capacity, low leakage power, and data persistence make it a promising candidate for enhancing ML capabilities atthe edge. However, integrating PCM into edge devices isnot without challenges, particularly concerning (slow) write performance and endurance. To address these obstacles, researchers have suggested using various PCM write modes that provide a balance between write latency and data retention. For instance, fast write mode significantly reduces write latency and energy consumption, but due to low retention time, it demands data refreshing. Addressing these trade-offs is essential for optimizing the role of PCM in edge ML applications. Another NVM technology widely used in edge devices is Ferroelectric RAM (FeRAM), which offers fast access speedsand large capacities suitable for edge contexts. The ability of FeRAM to be used as byte-addressable memory allows software to directly store important runtime information inside of the FeRAM, which can assist fast transfer between powersaving states of the system. Especially for timing predictable ML applications, minimal overhead transitions between powersaving states allow for energetic optimization while not affecting the timeliness of the system. On the downside, FeRAM is a read-destructive memory, which increases wear and shortens its lifespan, necessitating careful management in critical ML applications. Beyond edge ML applications, NVMs have also demonstrated potential in accelerating large-scale search operations, a common requirement in many machine learning tasks. The vast data scale and irregular memory access patterns involved in these operations present significant challenges. NVM-based content addressable memories (CAMs) have emerged as a promising compute in memory (CIM) search solution for these applications. However, selecting the optimal NVM devices and architectures for CAM-based accelerator is a complex task. Atthe architecture level, factors such as the size of the basic CAM array and the partition and merge schemes used to combine array results significantly influence application-level accuracy as well as the area, latency, and energy efficiency. Furthermore, read/write costs and device variability associated with different NVMs affect overall performance and accuracy. It is crucial to examine how various NVM device and architecture choices impact the performance of in-memory search accelerators. A thorough examination/research of how variou

FAQs

How do NVM technologies help edge machine learning workloads?

NVMs such as PCM and FeRAM offer larger capacity, low leakage power, and persistence. The paper highlights benefits for energy efficiency and latency reduction, while also addressing write performance and endurance constraints.

What trade-offs arise when using PCM for edge ML?

Fast PCM write modes reduce write latency and energy consumption, but require data refreshing due to low retention time. Slow write modes better preserve data longer but increase latency, so balancing both is essential.

Why are NVM-based CAM/CIM accelerators relevant for in-memory search?

Large-scale search uses vast data and irregular memory access patterns that traditional architectures struggle with. NVM-based CAMs enable compute-in-memory search, but device selection and CAM array partition/merge strategies significantly affect accuracy and efficiency.

Aug 2, 2026

Upload by Evangeline