[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84167-en":3,"doc-seo-84167-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84167,1374391974468,"Eden","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Multimodal Smart Glove for Sign Language Recognition Using Deep Learning","Sign language recognition systems aim to support inclusive communication between deaf or hard-of-hearing individuals and the broader public, yet many deployments struggle in real-world conditions. This work presents a deployable smart-glove solution that combines wearable sensing and deep learning for gesture understanding. Flex sensors and an IMU capture finger articulation and 3D hand motion, while a camera extracts facial cues. ESP32-C6 transmits sensor streams to an LSTM model to learn temporal dynamics, achieving about 95% accuracy and enabling real-time inference via TensorFlow Lite.","arXiv :2607 .06996v 1 [ cs .HC] 8 Jul 2026  \nMultimodal Smart Glove for Sign Language Recognition  \nUsing Deep Learning  \nAnh Thu Nguyen Ngoc 1 , Tam Phong Truong 1 , Thai Anh Nguyen Duong 1 , Vu Linh Nguyen2 , and Manh Duong Phung2  \n1 Fulbright University Vietnam, Ho Chi Minh City, Vietnam  \n2 College of Engineering and Computer Science and Smart Green Transformation Center (GREEN-X), VinUniversity, Hanoi, Vietnam  \nAbstract. Sign language recognition technologies can improve communication between deaf individuals and the broader community, but many existing systems face challenges in real-world deployment. This paper presents a deployable smart glove system for sign language recognition that integrates wearable sensing and deep learning. The glove incorporates flex sensors and an inertial measurement unit (IMU) to capture finger articulation and hand motion, while facial cues are obtained through a camera. Sensor data are transmitted via an ESP32-C6 microcontroller and processed using a long short-term memory (LSTM) network to model temporal gesture dynamics. Experimental results show that the proposed model achieves an overall recognition accuracy of approximately 95% . The trained model is further converted to TensorFlow Lite for real-time inference. This demonstrates the feasibility of the system for practical sign language translation applications.  \nKeywords: Vietnamese Sign Language (VSL), smart glove, gesture recognition, deep learning  \n1 Introduction  \nThe design and development of effective tools for translating sign language are increasingly necessary to promote inclusive communication between deaf or hard-of-hearing individuals and the broader community. Sign languages are complex visual-spatial languages that rely on hand movements, facial expressions, and body posture, which make real-time interpretation challenging for people who do not understand them [3] . As a result, communication barriers often limit access to education, healthcare, public services, and employment opportunities for the deaf community. Recent advances in artificial intelligence, computer vision, and sensor technologies provide new opportunities to automatically detect and interpret hand gestures for sign language translation.  \nTwo primary approaches have dominated the development of sign language translation systems: vision-based approaches and glove-based approaches. Vision-based approaches employ cameras together with computer vision and machine learning algorithms to interpret hand gestures directly from visual data [2, 12] . By analyzing video frames or image sequences, these systems detect and track hand movements, extract spatial and temporal features, and map the observed gestures to corresponding words or  \nsentences. With the rapid progress of deep learning, particularly convolutional neural networks [16] and transformer-based models [6], vision-based methods have achieved significant improvements in gesture recognition accuracy and robustness [1, 14] . However, these approaches often face practical limitations, including sensitivity to lighting variations, background clutter, occlusions, and changes in camera viewpoint. Advanced sensors, such as RGB-D cameras and LiDAR [8,9], can partially mitigate these issues. Nevertheless, continuous tracking of fine-grained hand movements remains computationally demanding and may require carefully controlled recording conditions.  \nThese challenges have motivated the exploration of glove-based approaches for sign language recognition. Unlike vision-based systems, glove-based solutions rely on wearable sensors to directly capture hand motion and finger articulation, enabling more stable and precise measurements for gesture interpretation [11, 13] . Such systems typically incorporate multiple sensing components such as flex sensors and inertial measurement units (IMUs) to measure finger bending, hand orientation, and motion dynamics [15] . Early work demonstrated that instrumented gloves coul","cbCaijjIoX4Hxu4X","https://ap.wps.com/l/cbCaijjIoX4Hxu4X","pdf",1195703,1,10,"English","en",105,"# Abstract\n# Introduction\n# System architecture","[{\"question\":\"What sensors are used in the proposed smart glove system?\",\"answer\":\"The glove uses flex sensors and an inertial measurement unit (IMU) to capture finger bending and hand motion. Facial cues are obtained through a camera in the same multimodal setup.\"},{\"question\":\"How are the sensor signals and facial information processed to recognize signs?\",\"answer\":\"Sensor data transmitted by an ESP32-C6 microcontroller are modeled with an LSTM network to learn temporal gesture dynamics, while facial information is fused with the wearable signals for gesture inference.\"},{\"question\":\"How does the system achieve real-time recognition in practice?\",\"answer\":\"After training, the model is converted to TensorFlow Lite to support real-time inference on the target execution environment.\"}]",1784193601,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"multimodal-smart-glove-for-sign-language-recognition-using-deep-learning","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/multimodal-smart-glove-for-sign-language-recognition-using-deep-learning/84167/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What sensors are used in the proposed smart glove system?","Question",{"text":74,"@type":75},"The glove uses flex sensors and an inertial measurement unit (IMU) to capture finger bending and hand motion. Facial cues are obtained through a camera in the same multimodal setup.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How are the sensor signals and facial information processed to recognize signs?",{"text":79,"@type":75},"Sensor data transmitted by an ESP32-C6 microcontroller are modeled with an LSTM network to learn temporal gesture dynamics, while facial information is fused with the wearable signals for gesture inference.",{"name":81,"@type":72,"acceptedAnswer":82},"How does the system achieve real-time recognition in practice?",{"text":83,"@type":75},"After training, the model is converted to TensorFlow Lite to support real-time inference on the target execution environment.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":21,"slug":132},"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]