X-TIME - An in-memory engine for accelerating machine learning on tabular data with CAMs

Aug 4, 2026

Upload by Logic

Structured, or tabular, data is the dominant format in many data-science workflows, yet tree-based machine learning remains challenging to accelerate due to irregular memory access patterns and inference latency limits on CPUs and GPUs. X-TIME presents an analog-digital in-memory architecture that implements an increased-precision analog CAM combined with a programmable network-on-chip to run state-of-the-art tree models such as XGBoost and CatBoost. Single-chip evaluation on 16nm technology reports 119× lower latency with 9740× higher throughput versus a leading GPU, at 19W peak power.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Introduction

Problem: latency and throughput limits for tree-based ML on tabular data

Prior work and gaps in hardware acceleration

Proposed approach: in-memory computing with analog CAMs

Target models and evaluation setup

X-TIME: An in-memory engine for accelerating machine learning on tabular data with CAMs Giacomo Pedretti􀀃y , John Moon􀀃 , Pedro Bruel􀀃 , Sergey Serebryakov􀀃 , Ron M. Roth􀀃 , Luca Buonanno􀀃 , Tobias Ziegler􀀃 , Cong Xu􀀃 , Martin Foltin􀀃 , Paolo Faraboschi􀀃 , Jim Ignowski􀀃 , and Catherine E. Graves􀀃 􀀃 Artiﬁcial Intelligence Research Lab (AIRL), Hewlett Packard Labs, Milpitas (CA) y giacomo.pedretti@hpe.com arXiv :2304 .0 1285v2 [ cs .LG] 5 Apr 2023 Abstract—Structured, or tabular, data is the most common format in data science. While deep learning models have proven formidable in learning from unstructured data such as images or speech, they are less accurate than simpler approaches when learning from tabular data. In contrast, modern tree-based Machine Learning (ML) models shine in extracting relevant information from structured data. An essential requirement in data science is to reduce model inference latency in cases where, for example, models are used in a closed loop with simulation to accelerate scientiﬁc discovery. However, the hardware acceleration community has mostly focused on deep neural networks and largely ignored other forms of machine learning. Previous work has described the use of an analog content addressable memory (CAM) component for efﬁciently mapping random forests. In this work, we focus on an overall analog-digital architecture implementing a novel increased precision analog CAM and a programmable network on chip allowing the inference of stateof-the-art tree-based ML models, such as XGBoost and CatBoost. Results evaluated in a single chip at 16nm technology show 119􀀂 lower latency at 9740􀀂 higher throughput compared with a stateof-the-art GPU, with a 19W peak power consumption. I. INTRODUCTION Extracting relevant information from structured (i.e., tabular) data is of utmost importance in practical data science. Tabular data consist of a matrix of samples organized in rows, each of which has the same set of features in its columns, and is commonly used for medical, ﬁnancial, scientiﬁc, and networking applications, enabling efﬁcient search operations on large-scale datasets. While deep learning has shown impressive improvements in accuracy for unstructured data, such as image recognition or machine translation, treebased machine learning (ML) models still outperform neural networks in classiﬁcation and regression tasks on tabular data [13], [21],[60] . This difference in model accuracy is due to robustness to uninformative features of tree-based models, bias towards a smooth solution of neural networks models, and in general the need for rotation invariant models where the data is rotation invariant itself, as in the case of tabular data [21] . With high model accuracy and ease of use, treebased ML models are greatly preferred by the data science community [7], with > 74% of data scientists preferring to use these models while < 40% choose neural networks in a recent survey. However, tree-based models have garnered little attention in the architecture and custom accelerator research ﬁelds, particularly compared to deep learning. While small models are easily executed, performance issues for larger tree-based ML models arise due to limited inference latency both on CPU and GPUs [26], [67] originating from irregular memory accesses and thread synchronization issues. Recent work attempted to address this limitation by improving GPU memory organization with compiler optimizations [4], [5] and model-tailored strategies for inference [67], with limited success. As large tree-based models are ﬁnding performance limits in existing hardware, larger models (>1M nodes) are demonstrating increasingly compelling uses in practical environments where high throughput and moderate latency are desired [1], [3] . For example, the winner of a recent IEEECIS Fraud detection competition used a tree-based ML model with 20M nodes [1] . Moreover, reducing inference latency is critical in scientiﬁc applications whe

FAQs

Why are tree-based machine learning models important for tabular data?

Tree-based models extract relevant information from structured, tabular datasets and tend to outperform neural networks on many classification and regression tasks. They are also preferred in practice because of accuracy and usability on this data type.

What is the key bottleneck when accelerating large tree-based ML models on existing hardware?

Inference latency and throughput limits arise from irregular memory accesses and thread synchronization issues, especially for larger trees with millions of nodes.

What does X-TIME contribute to accelerate tree-based ML inference?

X-TIME introduces an analog-digital in-memory design using an increased-precision analog CAM plus a programmable network-on-chip. It enables efficient inference of models like XGBoost and CatBoost directly in memory, reducing data movement and lowering latency.

Aug 4, 2026

Upload by Logic