Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference

Q: 为什么在边缘设备上研究VLM推理能耗很重要？

边缘端部署需要实时视觉推理，但VLM在实际设备上的能量足迹长期缺乏系统量化，导致效率优化方向可能偏离真实瓶颈。

Q: 该研究发现的主要能耗结论有哪些？

平均推理功耗对输入分辨率、图像复杂度和提示类型基本保持不变（<5%波动）。能耗差异主要来自推理时间变化，而输出token成本显著高于输入token。

Q: “真正的能耗瓶颈”指的是什么？

瓶颈不在模型看到的内容（视觉token处理），而在模型“说了多少”（输出token数量）。输出token在墙钟时间上比输入token高11–39×，并主导延迟与能耗。

Jul 16, 2026

Upload by Aurelia

Vision-Language Models (VLMs) underpin embodied AI, yet their on-edge energy costs are not well quantified. This work provides the first systematic energy profiling of on-device VLM inference across five models, four input resolutions, and two hardware platforms. Results show average inference power stays nearly constant (<5% change), while latency and energy vary mainly with inference time. Output tokens dominate cost: each output token takes 11–39× more wall-clock time than each input token. Image complexity affects energy up to 4.1× via output length, not visual compute, limiting visual token pruning to ~10% savings while controlling output length can save up to 97% energy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Abstract

Introduction

Embodied AI and the VLA Paradigm

Edge Deployment Constraints and Efficient VLMs

Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference Junfei Zhan Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia, PA, USA [zjf2024@engineering.upenn.edu](zjf2024@engineering.upenn.edu) Haoxun Shen Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia, PA, USA [haoxun@engineering.upenn.edu](haoxun@engineering.upenn.edu) Mingang Guo Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia, PA, USA [gmingang@engineering.upenn.edu](gmingang@engineering.upenn.edu) Zixuan Huang Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen, China [zx.huang5@siat.ac.cn](zx.huang5@siat.ac.cn) Tengjiao He College of Information Science and Technology Jinan University Guangzhou, China [htj2018@jnu.edu.cn](htj2018@jnu.edu.cn) arXiv :2607 .09520v1 [ cs .CV] 10 Jul 2026 Abstract Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual tokens, implicitly treating visual processing asthe dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX) . Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output token costs 11 to 39× more wall-clock time than each input token due to the compute-bound and memory-bound asymmetry between prefill and decode, making output token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1× energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual token pruning: even removing all visual tokens saves at most 10% of total energy for fixed-token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission [and/or a fee. Request permissions from permissions@acm.org](and/or a fee. Request permissions from permissions@acm.org). MM’26, Rio de Janeiro, Brazil © 2026 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN XXX-X-XXXX-XXXX-X/26/10 [https://doi.org/XX.XXXX/XXXXXXX.XXXXXXX](https://doi.org/XX.XXXX/XXXXXXX.XXXXXXX) Figure 1: Energy anatomy of VLM inference on the RTX 3070 at 448×448. (a) Decode dominates at 86–97%. (b) Each output token costs 11–39× more than each input token. bottleneck in edge VLM inference is not what the model sees, but how much it says. CCS Concepts • Computer systems organization → Embedded and cyberphysical systems; • Computing methodologies → Machine learning; • Hardware

FAQs

为什么在边缘设备上研究VLM推理能耗很重要？

边缘端部署需要实时视觉推理，但VLM在实际设备上的能量足迹长期缺乏系统量化，导致效率优化方向可能偏离真实瓶颈。

该研究发现的主要能耗结论有哪些？

平均推理功耗对输入分辨率、图像复杂度和提示类型基本保持不变（<5%波动）。能耗差异主要来自推理时间变化，而输出token成本显著高于输入token。

“真正的能耗瓶颈”指的是什么？

瓶颈不在模型看到的内容（视觉token处理），而在模型“说了多少”（输出token数量）。输出token在墙钟时间上比输入token高11–39×，并主导延迟与能耗。

Jul 16, 2026

Upload by Aurelia