[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82426-en":3,"doc-seo-82426-105":28,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":11,"language":21,"language_code":22,"site_id":23,"html_lang":22,"table_of_contents":24,"faqs":25,"seo_title":13,"seo_description":14,"update_tm":26,"read_time":27},82426,13056703020460,"Valentina","https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923",8,"Research & Report","Toward Real-Time Sentence-Level Sign Language Translation","Most sign language understanding systems rely on isolated sign recognition, which limits usability in natural communication. This work studies sentence-level sign language translation (SLT) aimed at real-time deployment. A SHuBERT–ByT5 translation stack is fine-tuned with QLoRA on a uniformly sampled 9,872-example subset of How2Sign (SHuBERT frozen). It achieves validation BLEU 16.7, test BLEU 15.9, and BLEURT 44.7. A hardware-aware streaming system with bounded queues, chunked ingestion, parallel perception, temporal reordering, and a sentence-boundary state machine reduces mean latency from 1.873 to 1.354s (27.71%) and P95 from 2.919 to 2.130s (27.03%).","Toward Real-Time Sentence-Level Sign Language Translation  \nThanh-Hoang Nguyen Doan  \nThe University of Danang – University of Science and Technology [102230150@sv1.dut.udn.vn](102230150@sv1.dut.udn.vn)  \narXiv :2607 .096 1 1v 1 [ cs .CL] 10 Jul 2026  \nAbstract  \nMost sign language understanding systems operate at the level of isolated signs, limiting their usefulness in natural communication. We study sentence-level sign language translation (SLT) with the primary goal of real-time deployment rather than proposing a new translation architecture. We fine-tune a SHuBERT–ByT5 translation stack on a uniformly sampled 9,872-example subset of How2Sign, selected because of compute and storage constraints, using QLoRA while keeping SHuBERT frozen. The model obtainsa validation BLEU of 16.7 and, on the test split, BLEU 15.9 and BLEURT 44 .7. The main contribution is a hardware-aware streaming system: a Raspberry Pi 4B reference client provides camera capture, local text display, and speech output, while compute-intensive perception and translation run on a CPU/GPU backend. The capture protocol remains client-agnostic, so the same backend can serve a browser, phone, or laptop. Chunked ingestion, bounded queues, parallelized perception, temporal reordering, and a sentence-boundary state machine reduce mean post-finalization response latency from 1.873 to 1.354 seconds (27.71%) and P95 latency from 2.919 to 2.130 seconds (27.03%) over the complete 9,872-example working subset.  \n1 Introduction  \nSign languages are complete natural languages with their own lexicon, grammar, and non-manual grammatical markers. Computational work on sign languages has historically concentrated on isolated sign language recognition (ISLR), in which a short, trimmed clip is mapped to a single gloss or word. This framing is convenient for data collection, annotation, training, and evaluation, because every example contains exactly one linguistic unit, allowing a model to focus on discriminative visual cues such as hand shape, hand location, movement direction, facial expression, and body posture. However, a single sign is a coarse unit of meaning: in isolation it often fails to carry the speaker’s full communicative intent.  \nIn real interaction, signers do not produce discrete words separated by clean boundaries. They produce continuous streams of signs in which movements are chained together, co-articulated, and modulated in rhythm. For this reason we shift the target from word-level recognition to sentencelevel sign language translation (SLT), where the input isan entire continuous utterance and the output is a complete text sentence (Camgöz et al., 2018, 2020) . Operating atthe sentence level lets the model learn contextual relations between signs, resolve ambiguities that arise when a sign appears alone, and produce output that can be shown as text or spoken directly, rather than a sequence of gloss labels.  \nThis shift comes at a cost. Sentence-level SLT must process longer video with more frames, contend with fuzzy  \nboundaries between signs, and rely on richer supervision. Non-manual channels—facial expression, mouthing, and upper-body motion—become obligatory signals rather than optional cues; a model that attends only to the hands discardsa substantial part of the linguistic information.  \nDeployment adds a further constraint. A useful assistive system should respond quickly, yet translating each sign the instant it appears is both error-prone and unnatural. We therefore adopt an utterance-then-translate design: the system continuously ingests a signing stream, detects when an utterance ends, and emits a translation shortly afterwards. The engineering goal is to balance the contextual accuracy of a sentence-level model against a response latency low enough for near real-time communication.  \nTo realize this design we build on SHuBERT (Gueuwou et al., 2025), a self-supervised model that learns sign representations from roughly a thousand hours of sign","cbCaibMScjGRl3ql","https://ap.wps.com/l/cbCaibMScjGRl3ql","pdf",234934,1,"English","en",105,"# Abstract\n# Introduction\n## Problem: isolated sign recognition limitations\n## Sentence-level SLT objective\n## Challenges and deployment constraints\n## System design and hardware-aware streaming contributions","[{\"question\":\"Why move from isolated sign recognition to sentence-level translation?\",\"answer\":\"Isolated sign recognition maps short clips to single glosses or words, but a single sign cannot capture full communicative intent. Sentence-level SLT processes continuous utterances and learns contextual relations to resolve ambiguities and output complete sentences.\"},{\"question\":\"How is the translation model fine-tuned for deployment?\",\"answer\":\"The work fine-tunes a SHuBERT–ByT5 stack using QLoRA on a uniformly sampled 9,872-example subset of How2Sign, while keeping the SHuBERT encoder frozen to fit compute and storage constraints.\"},{\"question\":\"What engineering techniques reduce response latency in the streaming system?\",\"answer\":\"The system uses chunked ingestion with bounded queues, parallelized perception, temporal reordering, and a sentence-boundary state machine. These reduce mean post-finalization latency from 1.873 to 1.354 seconds and P95 from 2.919 to 2.130 seconds.\"}]",1784180322,20,{"code":4,"msg":29,"data":30},"ok",{"site_id":23,"language":22,"slug":31,"title":13,"keywords":32,"description":14,"schema_data":33,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":26},"toward-real-time-sentence-level-sign-language-translation","",{"@graph":34,"@context":84},[35,52,67],{"@type":36,"itemListElement":37},"BreadcrumbList",[38,42,46,49],{"item":39,"name":40,"@type":41,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":43,"name":44,"@type":41,"position":45},"https://docshare.wps.com/document/","Document",2,{"item":47,"name":12,"@type":41,"position":48},"https://docshare.wps.com/document/research-report/",3,{"item":50,"name":13,"@type":41,"position":51},"https://docshare.wps.com/document/toward-real-time-sentence-level-sign-language-translation/82426/",4,{"url":50,"name":13,"@type":53,"author":54,"headline":13,"publisher":56,"fileFormat":59,"inLanguage":22,"description":14,"dateModified":60,"datePublished":61,"encodingFormat":59,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":55},"Person",{"url":39,"name":57,"@type":58},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":20},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why move from isolated sign recognition to sentence-level translation?","Question",{"text":74,"@type":75},"Isolated sign recognition maps short clips to single glosses or words, but a single sign cannot capture full communicative intent. Sentence-level SLT processes continuous utterances and learns contextual relations to resolve ambiguities and output complete sentences.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How is the translation model fine-tuned for deployment?",{"text":79,"@type":75},"The work fine-tunes a SHuBERT–ByT5 stack using QLoRA on a uniformly sampled 9,872-example subset of How2Sign, while keeping the SHuBERT encoder frozen to fit compute and storage constraints.",{"name":81,"@type":72,"acceptedAnswer":82},"What engineering techniques reduce response latency in the streaming system?",{"text":83,"@type":75},"The system uses chunked ingestion with bounded queues, parallelized perception, temporal reordering, and a sentence-boundary state machine. These reduce mean post-finalization latency from 1.873 to 1.354 seconds and P95 from 2.919 to 2.130 seconds.","https://schema.org",{"og:url":50,"og:type":86,"og:title":13,"og:site_name":57,"og:description":14},"article",{"robots":88,"canonical":50},"index,follow",{"doc_id":7,"site_id":23},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":44,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":45,"doc_module":4,"doc_module_name":44,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":51,"doc_module":4,"doc_module_name":44,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":44,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":44,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":44,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":44,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":44,"category_name":124,"show_sort_weight":27,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":27,"doc_module":4,"doc_module_name":44,"category_name":127,"show_sort_weight":27,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":44,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":44,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]