[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-273424-105":59,"doc-detail-273424-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","audio-interaction-model-audio-interaction-soundflow-streamaudio-2m","Audio Interaction Model - Audio-Interaction, SoundFlow, StreamAudio-2M","","Audio is continuous and interactive, while most Large Audio Language Models (LALMs) operate offline and streaming systems typically focus on ASR or dialogue. This work defines an always-on perceive–decide–respond Audio Interaction Model that tracks context, determines when intervention is needed, and generates responses without stopping listening. It introduces Audio-Interaction and SoundFlow with streaming-native data construction, comprehension-aware silence/response supervision, dual-loss training, and asynchronous FIFO inference. A new StreamAudio-2M corpus and Proactive-Sound-Bench enable evaluation across multiple interaction behaviors.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/audio-interaction-model-audio-interaction-soundflow-streamaudio-2m/273424/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/audio-interaction-model-audio-interaction-soundflow-streamaudio-2m/273424.png","ImageObject",300,407,{"name":92,"@type":93},"Patrick","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-11","2026-09-15",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",13,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What problem does the Audio Interaction Model address?","Question",{"text":113,"@type":114},"It bridges the gap between strong offline audio LALMs and narrow real-time systems by enabling an always-on streaming loop that continuously listens and decides when to respond.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does the model decide when to intervene?",{"text":118,"@type":114},"The decide stage uses comprehension-grounded triggers, learning from explicit silence/response labels and comprehension-aware supervision rather than relying only on shallow acoustic cues.",{"name":120,"@type":111,"acceptedAnswer":121},"What components support real-time operation during training and inference?",{"text":122,"@type":114},"SoundFlow constructs coherent streaming-native data, applies time-frequency joint preprocessing to reduce artifacts, uses history-review training for long-range context, and employs asynchronous FIFO scheduling to decouple encoding from generation.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},273424,1789471378,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},549758146520,"https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470","Audio Interaction Model  \narXiv : 2606 .05 12 1v2 [cs .SD] 2 1 Aug 2026  \nZhifei Xie 1 ∗ , Zihang Liu2* , Ze An2 , Xiaobin Hu2 , Yue Liao2 , Ziyang Ma 1 , Dongchao Yang3 , Mingbao Lin2†, Deheng Ye 1†, Shuicheng Yan2†, Chunyan Miao 1†  \n1NTU 2NUS 3 CUHK  \n[Zhifei001@e.ntu.edu.sg](Zhifei001@e.ntu.edu.sg)  \n Project page: [https://xzf-thu.github.io/Audio-Interaction](https://xzf-thu.github.io/Audio-Interaction)  \n [Data:](Data: huggingface.co/datasets/zhifeixie/StreamAudio-2M)[ huggingface.co/datasets/zhifeixie/StreamAudio-2M](Data: huggingface.co/datasets/zhifeixie/StreamAudio-2M)  \nAbstract  \nAudio is continuous and interactive, yet most Large Audio Language Models (LALMs) remain offline and streaming systems usually specialize in ASR or spoken dialogue. We formalize the Audio Interaction Model, an always-on perceive– decide–respond paradigm that tracks context, decides whether intervention is warranted, and responds without stopping listening. We instantiate it with Audio-Interaction and introduce SoundFlow, coupling streaming-native data construction, comprehension-aware silence/response supervision, dual-loss training, and asynchronous FIFO inference. We also construct StreamAudio-2M, a 2.6M-item, 302k-hour corpus spanning 7 capability families and 28 sub-tasks, together with Proactive-Sound-Bench. Across 8 benchmarks, AudioInteraction remains competitive on mainstream audio tasks while enabling spoken-instruction robustness, long-stream interaction, and proactive intervention.  \nIntroduction  \nAudio is a continuous channel through which people perceive events, infer intent, and decide when to act. A useful system should therefore do more than wait for a recording to end. It should listen as sound unfolds, retain context across time, and decide whether the current moment calls for a response. Large audio language models (LALMs) have made rapid progress toward general audio understanding (Chu et al. 2023; Tanget al. 2023; Xie and Wu 2024a), including speech recognition, audio question answering, emotion understanding, and reasoning over acoustic scenes. Yet most of these models inherit the offline interface of multimodal LLMs: given a complete audio clip and an instruction, produce one output.  \nThis interface is poorly matched to real acoustic interaction. In daily use, a model may need to transcribe speech while it is still being spoken, answer a follow-up question after several turns, ignore irrelevant background sounds, or proactively warn about a dangerous event before the user asks. Existing streaming systems address pieces of this problem, but they remain fragmented. Streaming ASR models focus on transcription; spoken-dialogue systems focus on conversational speech; full-duplex models reduce turn-taking latency but usually treat non-speech events as background. Thus, the field faces a gap between general audio understanding, where  \n∗Equal contribution. Work done during Zihang Liu ([liuzihang99@gmail.com](liuzihang99@gmail.com))’s visit at NUS-LV.†Corresponding authors.  \noffline LALMs are strong, and real-time audio interaction, where current systems remain narrow.  \nAs shown in Figure 1, we close this gap by formalizing the Audio Interaction Model. Instead of mapping a finished clip to a single answer, it performs an always-on perceive–decide–respond loop. Perceive continuously updates the acoustic and interaction context as new audio arrives; decide predicts whether accumulated context warrants silence or intervention; and respond generates content when triggered, without stopping subsequent perception. This unifies response timing and response content. Conventional tasks such as ASR, speech translation, audio understanding, and spoken dialogue become special cases, while simultaneous interpretation, audio instruction following, long-form interaction, and proactive assistance become natural extensions.  \nRealizing this loop introduces two coupled challenges absent from offline LALMs. (C1) Comprehension-grounded resp","cbCaic8Jaq8aixs5","https://ap.wps.com/l/cbCaic8Jaq8aixs5","pdf",13376009,18,"English","# Abstract\n# Introduction\n## Audio interaction gap\n## Audio Interaction Model\n## Challenges for real-time loop\n# Audio-Interaction and SoundFlow\n## Perceive\n## Decide\n## Respond\n# Dataset and benchmarks","[{\"question\":\"What problem does the Audio Interaction Model address?\",\"answer\":\"It bridges the gap between strong offline audio LALMs and narrow real-time systems by enabling an always-on streaming loop that continuously listens and decides when to respond.\"},{\"question\":\"How does the model decide when to intervene?\",\"answer\":\"The decide stage uses comprehension-grounded triggers, learning from explicit silence/response labels and comprehension-aware supervision rather than relying only on shallow acoustic cues.\"},{\"question\":\"What components support real-time operation during training and inference?\",\"answer\":\"SoundFlow constructs coherent streaming-native data, applies time-frequency joint preprocessing to reduce artifacts, uses history-review training for long-range context, and employs asynchronous FIFO scheduling to decouple encoding from generation.\"}]","Audio Interaction Model - Audio-Interaction, SoundFlow, StreamAudio-2M | PDF",45]