Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

Jul 16, 2026

Post-training quantization (PTQ) reduces memory and enables efficient deployment, yet low-bit PTQ (2-bit/4-bit) can cause large accuracy drops. Existing approaches typically reconstruct quantization errors using an unconstrained full-precision (FP) model. This work argues that low-bit PTQ is limited both by error minimization and by the FP model’s quantization-error tolerance. It introduces Efficient Tuning Before Quantization (ETBQ), a pre-conditioning SGD-tuning stage using perturbations from weight/activation error distributions, improving robustness without fake-quantized QAT training.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Background: PTQ vs. QAT

Existing PTQ methods and their limitations

Proposed idea: quantization-friendly FP models

Method: ETBQ Framework

Pre-quantization tuning via error-distribution perturbations

Stabilization with stochastic weight averaging

Experiments and Results

Benchmarks across vision tasks

Gains under W2A4 settings

Contributions and Positioning

Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models Peng Xia, Junbiao Pang*, Muhammad Ayub Sabir arXiv :2607 . 11359v1 [ cs .CV] 13 Jul 2026 Abstract—Post-training quantization (PTQ) compresses deep neural networks for deployment under limited memory and computational budgets. However, low-bit (i.e., 2-bit or 4-bit) PTQ often suffers from substantial performance degradation. Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction. We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of a FP model itself. In this paper, we propose Efficient Tuning Before Quantization (ETBQ), a pre-conditioning tuning stage for Stochastic Gradient Descent (SGD)-optimized models before PTQ. During tuning, the FP model is optimized under perturbations sampled from the error distributions of weight and activation quantization, guiding the model toward a loss-landscape region that is less sensitive to the subsequent PTQ. Unlike QAT, ETBQ does not train a fake-quantized deployment model, which is computationally and memory intensive. Instead, ETBQ outputs a FP model that can be used by any PTQ backend. Experiments on CIFAR- 100, Tiny-ImageNet, ImageNet, and Cityscapes provide consistent evidence that ETBQ improves low-bit PTQ across diverse tasks. Under W2A4 settings, e.g., ETBQ improves over naive PTQ by 2.14% top-1 accuracy on Tiny-ImageNet and by 5.80% mIoU on Cityscapes. Code is available at [https://github.com/](https://github.com/)[ ](https://github.com/)xpxpxp2001xpxpxp/ETBQ. Index Terms—Post-training quantization, low-bit quantization, model compression, quantization robustness, loss landscape. I. INTRODUCTION THE deployment of Deep Neural Networks (DNNs) on edge devices has made model compression a critical challenge in efficient deep learning. Among compression techniques, model quantization [1], [2] is particularly attractive, as it reduces memory usage and enables low-precision integer computation. Post-Training Quantization (PTQ) [3]–[5] is an efficient paradigm that avoids full-dataset retraining required by Quantization-Aware Training (QAT) [6]–[8] . Despite its efficiency, PTQ suffers significant accuracy degradation at low-bit widths (i.e., 2-bit or 4-bit) when both weights and activations are quantized. Existing PTQ methods mainly address accuracy degradation by mimicking the behavior of FP models. Representative approaches include learned weight rounding [3], block-wise output reconstruction [1], [9], stochastic activation perturbation [4], and prediction-difference-guided calibration [10] . * Corresponding author(s) . Peng Xia, Junbiao Pang, and M. Sabir are with the School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China (e-mail: junbiao [pang@bjut.edu.cn](pang@bjut.edu.cn)) Fig. 1. Comparison between Hessian spectral densities of ResNet-18 on CIFAR-100 . (a) Standard SGD model, with W2A4 accuracy of 76.47%(b) ETBQ-preconditioned model, with W2A4 quantization. ETBQ shifts the Hessian spectrum toward smaller eigenvalues, indicating a flatter loss basin and improved tolerance to low-bit quantization errors. More recent extensions sequentially reduce activation and weight errors [11] or recover quantization-induced information via lightweight compensation modules [12] . A fundamental question overlooked by conventional PTQ paradigms is: What intrinsic properties of FP models make them quantizationfriendly for low-bit PTQ? An intuitive yet largely underexplored hypothesis is that FP models must be robust to quantization-induced errors. We hypothesize that FP models residing in flatter loss basins are more likely to be quantization-friendly. To examine this hypothesis, we compare a standard SGD-optimized ResNet- 18 with its ET

FAQs

What problem does PTQ face when moving to very low bit widths?

Low-bit PTQ (e.g., 2-bit or 4-bit) often leads to substantial performance degradation compared with full-precision baselines.

How does ETBQ differ from Quantization-Aware Training (QAT)?

ETBQ performs a pre-conditioning tuning stage on the full-precision model using sampled quantization-error perturbations, without training a fake-quantized deployment model like QAT.

What types of evidence show ETBQ improves low-bit PTQ accuracy?

Experiments on CIFAR-100, Tiny-ImageNet, ImageNet, and Cityscapes consistently show improvements; for example, ETBQ improves over naive PTQ by 2.14% top-1 on Tiny-ImageNet and by 5.80% mIoU on Cityscapes under W2A4 settings.

Jul 16, 2026

Upload by Genevieve