BiSCo-LLM Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression

Jul 16, 2026

Large language models face deployment constraints from memory capacity, weight bandwidth, and checkpoint storage, making extreme low-bit compression a key requirement. Existing low-bit approaches either limit representation near 2 bits per weight or rely on vector-quantization codebooks that add lookup and extra storage accounting. BISCO-LLM introduces a codebook-free binary spherical coding framework using binarized spherical codes, a residual BSQ stage for rate–distortion control without stored codebooks, and category-wise recovery distillation for Transformer modules.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Introduction

BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression Yuantian Shao 1,2,†, Peisong Wang2,†,* , Zhilei Liu2 , Chuangyi Li2 , Yuanteng Chen2 , Pengcheng Xie3 , Yiwu Yao3 , Zhihui Wei 1,* , and Jian Cheng2,* arXiv :2607 .08643v 1 [ cs .LG] 9 Jul 2026 Abstract—Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainly follow two directions. Scalar or group-wise quantization is simple and compatible with efficient low-precision kernels, but its representation capacity becomes limited when the target budget approaches 2 bits per weight. Vector-quantized weight compression provides a richer block-level representation, but usually introduces explicit codebooks, index lookup, and additional storage accounting. This paper presents BISCO-LLM, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression. The proposed pipeline is built on three components. First, local weight chunks are mapped onto a unit hypersphere and binarized into compact spherical codes, so that the main payload is a bit-packed sign stream rather than explicit VQ centroids. Second, a residual BSQ stage encodes the reconstruction error left by the base spherical codec, providing an explicit rate–distortion path without stored codebooks. Third, category-wise recovery distillation is performed after replacing each Transformer module category, reducing themismatch between local weight reconstruction and assembled model behavior. A small 8-bit protected-channel path is used as an auxiliary stabilization mechanism for sensitive channelsand is counted separately from the BSQ payload. The reported storage budget includes binary codes, neural decoders, protectedchannel payloads, LoRA adapters, and metadata. On Qwen3-8B, BISCO-LLM obtains a WikiText-2 perplexity of 10.18 compared with 9.73 for the FP16/BF16 model, and an average downstream accuracy of 68.05 compared with 69.92 over the reported seventask evaluation set. These results indicate that codebook-free spherical coding can preserve model behavior under an extreme low-bit storage budget when residual coding, sensitivity-aware protection, and recovery distillation are jointly considered. Index Terms—Large language model compression, low-bit quantization, binary spherical coding, codebook-free compression, vector quantization, residual coding, LoRA compensation. I. INTRODUCTION Large language models (LLMs) have become foundation components for language understanding, reasoning, genera tion, and general-purpose AI services [1]–[3] . Despite their †Yuantian Shao and Peisong Wang contributed equally to this work. *Peisong Wang, Zhihui Wei, and Jian Cheng are corresponding authors. 1Yuantian Shao and Zhihui Wei are with Nanjing University of Science and Technology, Nanjing, China. 2Yuantian Shao, Peisong Wang, Zhilei Liu, Chuangyi Li, Yuanteng Chen, and Jian Cheng are with the Institute of Automation, Chinese Academy of Sciences, Beijing, China. 3Pengcheng Xie and Yiwu Yao are with Huawei, China. effectiveness, the continuous growth of model scale has introduced substantial deployment challenges. Recent technical reports show that LLM deployment targets have moved from several-billion-parameter models to hundreds-of-billions and trillion-scale architectures. DeepSeek-V3 and DeepSeek-R1, for example, adopt a 671B-parameter MoE architecture with 37B activated parameters per token; Kimi K2 scales this regime to a 1T-parameter MoE model with 32B activated parameters; and GLM-4.5 reports 355B total parameters with 32B activated parameters [4]–[7] . These models correspond to terabyte-scale BF16/FP16 checkpoint storage before KV caches, runtime buffers, and serving-system overhead are considered. As a result, efficient inference is constrained not only by peak arithmetic throughput, but also by the amount

FAQs

What problem does BiSCo-LLM address in LLM deployment?

It targets deployment bottlenecks caused by limited memory capacity, weight bandwidth, and checkpoint storage, especially under extreme low-bit weight budgets.

How does BISCO-LLM avoid using explicit codebooks?

It maps local weight chunks to a unit hypersphere and converts them into compact binary spherical codes, so the payload is a bit-packed sign stream rather than stored VQ centroids.

What techniques does BISCO-LLM use to improve reconstruction quality and model behavior?

It adds a residual BSQ stage to encode reconstruction error with an explicit rate–distortion path, and applies category-wise recovery distillation after updating each Transformer module category.

Jul 16, 2026

Upload by Valentina