[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82586-en":3,"doc-seo-82586-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82586,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","NPUsper Eliminating Redundant Computation for Real-Time Whisper on Mobile NPUs","NPUsper is a live transcription system that makes Whisper efficient on mobile NPUs by removing redundant computation in streaming inference. It detects hallucinated tokens online using temporal patterns in decoder cross-attention, avoiding heavy padding and limiting carryover by processing short audio with minimal overlap. For mobile execution, it proposes controlled unrolling via K-step chunk graphs to reduce KV-cache work and graph-dispatch overhead. Experiments show up to 4.84× lower per-word latency, 33.2× lower TTFT, and 88.64% lower average power, with comparable accuracy.","arXiv :2607 .0 1 108v 1 [ cs . SD] 1 Jul 2026  \nNPUsper: Eliminating Redundant Computation for Real-Time Whisper on Mobile NPUs  \nSihyeon Lee 1 Hojeong Lee2 Sungwon Woo 1  \nChengpo Yan2 Suman Banerjee2 ,† Seyeon Kim 1 ,†  \n1 Korea University 2University of Wisconsin–Madison  \n[xl6047@korea.ac.kr](xl6047@korea.ac.kr) [hojeong.lee@wisc.edu](hojeong.lee@wisc.edu) [vpfkfl753@gmail.com](vpfkfl753@gmail.com)[ ](vpfkfl753@gmail.com)[cyan46@wisc.edu](cyan46@wisc.edu) [suman@cs.wisc.edu](suman@cs.wisc.edu) [seyeon625@korea.ac.kr](seyeon625@korea.ac.kr)  \nAbstract  \nWe present NPUsper1 , a live transcription system that makes Whisper efficient on mobile NPUs by eliminating redundant computation. To avoid the heavy padding used by prior streaming systems, NPUsper detects hallucinated tokens online from temporal patterns in decoder cross-attention, allowing each inference round to process short audio inputs with minimal carryover. For efficient mobileNPU execution, we propose controlled unrolling, which executes autoregressive decoding as K-step chunk graphs, removing unnecessary KV-cache computation and reducing graph-dispatch overhead. NPUsper achieves up to 4.84× lower per-word latency, up to 33.2× lower time-to-first-token (TTFT), and up to 88.64% lower average power consumption compared with baselines, while maintaining comparable transcription accuracy. The code is available at [https://github](https://github) .  \ncom/npusper/NPUsper.  \n1 Introduction  \nOn-device AI inference is becoming increasingly important for mobile applications due to its advantages in privacy, latency, and reliability. Among these applications, real-time speech transcription has emerged as a key capability for intelligent assistants and accessibility services, where timely and accurate transcription is critical to user experience [Apple, Labiausse et al., 2025] . Enabling such functionality directly on mobile devices is particularly desirable, as it avoids the latency and connectivity limitations of cloud-based processing while preserving user privacy.  \nRecent systems increasingly rely on large foundation models for robust transcription, and Whisper [Radford et al., 2023] has emerged as a de facto standard due to its strong accuracy and scalability. As a result, there is growing interest in running Whisper directly on mobile devices for real-time, on-device transcription [Macháek et al., 2023, Wang et al., 2025, 2024, Macháek and Polák, 2025] . To meet the stringent latency and energy requirements of real-time processing, modern mobile platforms are equipped with Neural Processing Units (NPUs) that offer high-throughput and energy-efficient inference. However, fully leveraging mobile NPUs for real-time Whisper remains challenging, due to fundamental mismatches between Whisper’s streaming inference pipeline and the execution characteristics of NPUs.  \nFirst, existing streaming approaches for Whisper incur substantial redundant computation. Whisper is prone to hallucinations when processing short audio inputs that deviate from its 30-second training distribution [Wang et al., 2025] . To mitigate this, prior systems rely on padding and overlapping audio buffers, effectively reprocessing previously seen inputs to stabilize decoding as illustrated  \n†Corresponding authors.  \n1NPUsper combines “NPU” and “Whisper,” with “NP” also referring to no padding to avoid padding overhead.  \nPreprint.  \nRound Whisper input: fixed 30s Whisper input: 10-15 s + hush word Input: mostly new  \nRepeated inference Padding overhead  \n(a) Whisper-Streaming, SimulStreaming, and SimulWhisper  \nRepeated inference  \n(b) WhisperFlow  \nRepeated inference  \n(c) Ours  \nFigure 1: Redundant computation in existing systems. Existing systems construct each Whisper input by buffering previously processed audio with newly arrived audio and padding it to a fixed 30-second window to mitigate hallucinations. WhisperFlow [Wang et al., 2025] reduces padding overhead by replacing long padding with a trai","cbCaimFl6XrJbnc2","https://ap.wps.com/l/cbCaimFl6XrJbnc2","pdf",1887859,1,20,"English","en",105,"# Abstract\n# Introduction\n# Background and Related Work\n## Live Transcription Systems with Whisper","[{\"question\":\"What problem does NPUsper address in running Whisper on mobile NPUs?\",\"answer\":\"NPUsper targets two inefficiencies: redundant computation caused by padding/overlap in streaming Whisper, and a mismatch between Whisper’s dynamic autoregressive decoding (KV-cache growth) and mobile NPU static graph execution.\"},{\"question\":\"How does NPUsper reduce padding overhead during streaming inference?\",\"answer\":\"It detects hallucinated tokens online from temporal patterns in decoder cross-attention, enabling inference to process short audio inputs with minimal carryover instead of padding to a fixed 30-second window.\"},{\"question\":\"What performance improvements does NPUsper achieve compared with baselines?\",\"answer\":\"NPUsper reports up to 4.84× lower per-word latency, up to 33.2× lower time-to-first-token (TTFT), and up to 88.64% lower average power consumption while maintaining comparable transcription accuracy.\"}]",1784181650,50,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"npusper-eliminating-redundant-computation-for-real-time-whisper-on-mobile-npus","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/npusper-eliminating-redundant-computation-for-real-time-whisper-on-mobile-npus/82586/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does NPUsper address in running Whisper on mobile NPUs?","Question",{"text":75,"@type":76},"NPUsper targets two inefficiencies: redundant computation caused by padding/overlap in streaming Whisper, and a mismatch between Whisper’s dynamic autoregressive decoding (KV-cache growth) and mobile NPU static graph execution.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does NPUsper reduce padding overhead during streaming inference?",{"text":80,"@type":76},"It detects hallucinated tokens online from temporal patterns in decoder cross-attention, enabling inference to process short audio inputs with minimal carryover instead of padding to a fixed 30-second window.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance improvements does NPUsper achieve compared with baselines?",{"text":84,"@type":76},"NPUsper reports up to 4.84× lower per-word latency, up to 33.2× lower time-to-first-token (TTFT), and up to 88.64% lower average power consumption while maintaining comparable transcription accuracy.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":28,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]