HTF-MPR: A Heterogeneous TensorFlow Mapper Targeting Performance using Genetic Algorithms and Gradient Boosting Regressors

Jul 6, 2026

Upload by Aurelia

HTF-MPR introduces a performance-focused heterogeneous TensorFlow mapping framework that optimizes how devices are assigned to TensorFlow operations in distributed training. The method uses a genetic algorithm to search for mappings beyond TensorFlow’s default mapper, while Gradient Boosting Regressors provide a fitness predictive model to expand the search space. Experiments on ALEXNET, MNIST softmax, and VGG-16 show overall training-stage speedups of 1.18, 3.33, and 1.13, respectively.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

Introduction

TensorFlow computational graphs and mapping

Limits of default mapping

Alternative mapping strategies

HTF-MPR: A Heterogeneous TensorFlow Mapper Targeting Performance using Genetic Algorithmsand Gradient Boosting Regressors Ahmad Albaqsami, Maryam S. Hosseini, Nader Bagherzadeh Department of Electrical Engineering and Computer Science, University of California, Irvine, CA, USA Email: {aalbaqsa,mseyyedh,[nader](nader}@uci.edu)[}](nader}@uci.edu)[@uci.edu](nader}@uci.edu) Abstract—TensorFlow [1] is a library developed by Google to implement Artiﬁcial Neural Networks using computational dataﬂow graphs. The neural network has many iterations during training. A distributed, parallel environment is ideal to speedup learning. Parallelism requires proper mapping of devices to TensorFlow operations. We developed HTF-MPR framework for that reason. HTF-MPR utilizes a genetic algorithm approach to search for the best mapping that outperforms the default Tensorﬂow mapper. By using Gradient Boosting Regressors to create the ﬁtness predictive model, the search space is expanded which increases the chances of ﬁnding a solution mapping. Our results on well-known neural network benchmarks, such as ALEXNET, MNIST softmax classiﬁer, and VGG-16, show an overall speedup in the training stage by 1.18, 3.33, and 1.13, respectively. I. INTRODUCTION Machine Learning (ML) algorithms [2] have found a large number of applications in computer vision, data tracking, recommender systems, search engines and Artiﬁcial Intelligence (AI) in games. One notable advancement in ML algorithms is the use of Artiﬁcial Neural Networks (ANNs) [3] . ANNs are constructs that mimic how the brain works. In their most basic form, they consist of synapses and neurons, where the synapses are the weights and neurons are the functions (see Fig 1) . Companies invest in improving and utilizing ANNs for different tasks [4], leading to many applications applied to ANNs, creating deep and complex ANN architectures, ﬁnding techniques and accelerating the training and the inference of ANNs [5]–[7] . A number of softwares libraries have been developed to ease the construction of ANNs for end-users. One such library is TensorFlow [8]; a computational graph and numerical models library developed by Google. The application programming interface (API) makes it possible for data scientists to work with large models and many data samples in a distributed system without prior knowledge of the hardware architecture. The current state-of-the-art ANNs consist of hundreds of thousands of parameters, and require large data sets to train. The number of layers, features (inputs) and interconnections, result in a large number of parameters that require training, which prolongs the training process. Training in ANNs are iterative [9]; in each iteration, the process would require a feed-forward step through the ANN, and a back-propagation that ﬂows backwards. With each iteration, a set of data-samples (batch, or mini-batch) are fed ,ŝĚĚĞŶ >ĂǇĞƌ ϭ /ŶƉƵƚ ,ŝĚĚĞŶ >ĂǇĞƌ Ϯ KƵƚƉƵƚ ĂͿ 􀀄ƌƚŝĨŝĐŝĂů EĞƵƌĂů EĞƚǁŽƌŬ /ŶƉƵƚ ,ŝĚĚĞŶ ,ŝĚĚĞŶ KƵƚƉƵƚ >ĂǇĞƌ ϭ d& KƉĞƌĂƚŝŽŶ >ĂǇĞƌ Ϯ d& KƉĞƌĂƚŝŽŶ ďͿ dĞŶƐŽƌĨůŽǁ 'ƌĂƉŚ Fig. 1. Artiﬁcial Neural Network and its TensorFlow depiction to the ANN. This modiﬁes the parameters (weights and biases) which reduces a given loss-function. In TensorFlow, parameters, functions, and inputs are represented by computational graphs [1] . Computational graphs consist of edges and vertices in a Directed Acyclic Graph (DAG) . Edges carry multi-dimensional arrays known as tensors, and vertices are the functions, known as operations, applied to tensors. A simple translation from ANN to a TensorFlow computational graph is shown in Fig 1b. Speedup of these computational graphs is of importance. One such approach is to reduce the number of parameters ina ANN [10] . In [10], the authors compressed the ANN by pruning the number of neurons and synapses, which reduces the number of computations. However, this would slightly change the accuracy of the prediction model [10] .

FAQs

What problem does HTF-MPR address in TensorFlow?

HTF-MPR targets the need for proper mapping of heterogeneous devices to TensorFlow operations to accelerate distributed neural network training.

How does HTF-MPR search for an improved mapping?

It uses a genetic algorithm to explore mappings, guided by a fitness predictive model built with Gradient Boosting Regressors.

What speedups are reported on common benchmarks?

The reported overall speedups in the training stage are 1.18 for ALEXNET, 3.33 for MNIST softmax, and 1.13 for VGG-16.

Jul 6, 2026

Upload by Aurelia