The In-Car Sign Language Corpus (ICSL) : A Multi-Modal Resource for Constrained-Space Sign Language Recognition

Jul 16, 2026

This paper tackles the underexplored problem of using sign language recognition (SLR) in shared mobility vehicles, where the interior cabin creates constrained, occluded, and non-frontal conditions. The In-Car Sign Language (ICSL) dataset for Brazilian Sign Language (Libras) combines high-precision MoCap for an ideal baseline with real-world multi-modal recordings from a 2D camera and 3D Time-of-Flight sensors. Over 1.5 million synchronized frames are provided with gloss annotations for lexicals and non-lexicals, enabling training and evaluation of deep neural networks and future in-the-wild domain adaptation.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

Abstract

Introduction

arXiv :2607 . 1 134 1v 1 [ cs .CL] 13 Jul 2026 The In-Car Sign Language Corpus (ICSL): A Multi-Modal Resource for Constrained-Space Sign Language Recognition Raviteja Boddu∗ , Guilherme Vieira Leite†, Joed Lopes da Silva∗ ,Ângelo Benetti†, Isabela Barbieri†, Natália de Melo Afonso†, Thyago Santos‡, Helio Pedrini†, Felipe Venâncio Barbosa‡, José Mario De Martino†, Munir Georges∗ , Alessandro Zimmer∗ ∗Technische Hochschule Ingolstadt (THI), Bayern, Germany † Universidade Estadual de Campinas (UNICAMP), São Paulo, Brazil ‡ Universidade de São Paulo (USP), São Paulo, Brazil ∗ {raviteja.boddu, joed.lopesdasilva, munir.georges, [alessandro.zimmer}@thi.de](alessandro.zimmer}@thi.de)[ ](alessandro.zimmer}@thi.de)† [martino@unicamp.br](martino@unicamp.br),†[guilherme.leite@ic.unicamp.br](guilherme.leite@ic.unicamp.br) Abstract This paper addresses the challenges of using sign language within shared mobility services, such as taxis, carpools, or ride-sharing platforms. The use of sign language recognition (SLR) in real-world, confined environments, specifically vehicle interiors remains largely unexplored. To motivate research in this area, we present the In-Car Sign Language (ICSL) dataset for Brazilian Sign Language (Libras), with the long-term goal of improving public transport accessibility for the Deaf and Hard-of-Hearing community. The dataset consists of: (1) high-precision laboratory motion capture (MoCap) data to establish an idealized linguistic baseline and (2) real-world multi-modal in-car recordings captured using a 2D camera and 3D Time-of-Flight sensors. The dataset provides a basis for comparative analyses between synthesized signing avatar animations and recorded real signing interpreter videos, which enable future research into robust “in-the-wild” SLR models and domain adaptation. We describe in detail the use cases, the setup, the data collection protocol, and the metadata structure of the corpus. In total, we recorded a multimodal dataset exceeding 1 .5 million frames, comprising the synchronized multimodal streams described above featuring Libras users across various in-car scenarios. The corpus is provided with gloss annotation of lexical signs and non-lexical sign language elements specially designed to support the training and evaluation of deep neural networks for constrained space recognition. In-vehicle signing offers a technically significant example of a constrained, occluded, and non-frontal environment. While recognizing the diverse communication strategies already employed by the Deaf community, identifying automotive-specific limitations provides a useful stepping stone for research into enhancing in-car accessibility and passenger quality of life. Keywords: Brazilian Sign Language (Libras), Shared Mobility Service, Motion Capture (MoCap), In-Car Communication, Constrained Signing Space, Signing Avatars, Multimodal Sensors 1. Introduction The advancement in natural language processing and computer vision has brought us closer to seamless human-machine interaction. While the Deaf and Hard-of-Hearing (DHH) community has developed effective strategies for "on-the-go" communication, such as the use of mobile devices or visual cues, the interior of vehicles within shared mobility services represents a technically significant and under-researched environment for Sign Language Recognition (SLR) . Specifically, the car cabin serves as a critical case study for constrained, occluded, and nonfrontal signing environments, which are often overlooked in traditional laboratory-based datasets. While Brazilian Sign Language (Libras) has an established linguistic foundation, there is currently no specialized resource documenting how signing is produced and processed in the highly constrained physical and visual environments of a car cabin (dos Santos et al. , 2025 ; Lee et al. , 2025) . The relevant studies and corpora that exist for Libras are predominantly recorded in laboratorycontrolled setting

FAQs

What problem does the ICSL paper address?

The paper addresses the challenge of performing sign language recognition in real vehicle interiors used by shared mobility services, where space is constrained and visibility is frequently obstructed.

What data sources are included in the ICSL dataset?

ICSL includes high-precision laboratory motion capture (MoCap) data for an idealized linguistic baseline and real-world in-car multi-modal recordings using a 2D camera and 3D Time-of-Flight sensors.

What annotations and scale does the dataset provide for research?

The corpus is annotated with glosses for lexical signs and non-lexical elements, and it contains more than 1.5 million synchronized frames across multiple in-car scenarios to support training and evaluation of deep neural networks.

Jul 16, 2026

Upload by Clementine