Lyapunov Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

Jul 16, 2026

Upload by Hazel

Low-precision neural networks enable deployment on resource-constrained hardware, yet fixed-point arithmetic introduces hidden failure modes beyond idealised quantisation. Two’s-complement overflow wrapping can corrupt activations by flipping sign and magnitude, amplifying numerical error propagation and severely degrading accuracy. The work presents a Lyapunov-stabilised quantisation framework for hardware-style wrapping arithmetic, monitoring layerwise hidden-state energy and applying a monotone projection to enforce bounded, non-increasing state evolution across depth. Experiments on MNIST with a compact transformer show unconstrained wrapped QAT collapses to near-chance with >11% activation overflow, while the proposed projection suppresses overflow below 0.012% and restores stable learning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

Introduction

Low-precision arithmetic and quantisation benefits

Overflow failure modes in fixed-point hardware

Viewing deep models as layerwise dynamical systems

Lyapunov methods for stability and control

Proposed Lyapunov-stabilised quantisation framework

Layerwise Lyapunov energy monitoring

Monotone projection with bounded state energy

Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic Anis Hamadouche The School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK[anis.hamadouche@hw.ac.uk](anis.hamadouche@hw.ac.uk) Amir Hussain SDAIA-KFUPM Joint Research Centre for Artificial Intelligence King Fahd University of Petroleum and Minerals Dhahran, Saudi Arabia [amir.hussain@kfupm.edu.sa](amir.hussain@kfupm.edu.sa) arXiv :2607 .0453 1v 1 [ cs .LG] 5 Jul 2026 Abstract—Low-precision neural networks are attractive for resource-constrained hardware, but fixed-point arithmetic introduces failure modes that are often hidden by idealised quantisation models. In particular, two’s-complement overflow wrapping can corrupt hidden activations by changing both their magnitude and sign, leading to unstable numerical error propagation and severe accuracy degradation. This paper proposesa Lyapunov-stabilised quantisation framework for low-precision neural networks operating under hardware-style wrapping arithmetic. The hidden-state energy is monitored through a layerwise Lyapunov function, and a monotone projection is applied to enforce bounded and non-increasing state evolution across depth. The method is evaluated on MNIST using a compact patch-based transformer under post-training quantisation and quantisation-aware training with fixed-point bit-widths from 4 to 16 bits. Monte Carlo results show that unconstrained wrapped quantisation-aware training collapses to near-chance accuracy across 6–16 bits, with activation overflow rates exceeding 11%. In contrast, the proposed monotone Lyapunov projection suppresses activation overflow to below 0.012% and restores stable lowprecision learning, achieving 86.55% ± 0.65% accuracy at 12 bits. These results demonstrate that Lyapunov-based state control can act as a hardware-aware stabilisation mechanism for reliable fixed-point neural inference and training. Index Terms—Low-precision neural networks, quantisationaware training, fixed-point arithmetic, overflow wrapping, Lyapunov stability, neural network safety, hardware-aware machine learning, transformer networks. I. INTRODUCTION The deployment of deep neural networks on edge devices, embedded processors, field-programmable gate arrays, and custom accelerators has made low-precision arithmetic a central topic in hardware-aware machine learning. Quantisation reduces memory footprint, bandwidth, arithmetic cost, and energy consumption by representing weights, activations, or accumulators using fewer bits than conventional floatingpoint arithmetic. These benefits are particularly important for real-time and resource-constrained applications, where fullprecision inference may be too expensive in terms of latency, silicon area, or power consumption. Consequently, posttraining quantisation (PTQ) and quantisation-aware training (QAT) have become standard techniques for mapping neural networks onto efficient low-precision hardware [8, 11, 15] . Most quantisation studies focus on rounding error, clipping error, calibration, mixed precision, or the accuracy degradation caused by reducing the number of representable values. However, practical fixed-point hardware introduces an additional failure mode: arithmetic overflow. In saturating arithmetic, an overflowed value is clipped to the largest or smallest representable value. In wrapping arithmetic, which follows two’scomplement behaviour, an overflowed value wraps around the finite integer range. This means that a large positive value may become negative, and a large negative value may become positive. Such behaviour is efficient in hardware but can be numerically destructive. In quantised neural networks, overflow in activations or accumulators can corrupt hiddenstate trajectories and produce abrupt changes in the represented computation [4, 14] . This issue is especially important for deep residual and transformer-like architectures. These models can be

FAQs

What specific problem does fixed-point overflow wrapping create for quantised neural networks?

Wrapping overflow follows two’s-complement behaviour, so large positive values can become negative and vice versa. In quantised networks, this can abruptly corrupt hidden-state trajectories and propagate destabilising numerical errors through later layers.

How does the proposed Lyapunov-guided framework improve stability during fixed-point QAT/PTQ?

It monitors normalised hidden-state energy using a layerwise Lyapunov function and applies a monotone projection to enforce bounded, non-increasing energy across depth. This constrains states away from regions prone to wraparound.

What do the MNIST results indicate about unconstrained vs. Lyapunov-projected training?

Unconstrained wrapped quantisation-aware training collapses to near-chance accuracy across 6–16 bits, with activation overflow rates exceeding 11%. With the monotone Lyapunov projection, activation overflow drops below 0.012% and accuracy reaches 86.55% ± 0.65% at 12 bits.

Jul 16, 2026

Upload by Hazel