Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies

Jul 16, 2026

Autism spectrum disorder (ASD) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited. This study evaluates automated detection of autism-related self-stimulatory hand behaviors from video by optimizing both sequence-based neural architectures and temporal sampling rates. Long short-term memory (LSTM) and gated recurrent unit (GRU) models outperform prior CNN baselines, reaching peak accuracies near 97.5–98.75% at a 15-frame interval. The work further assesses ten augmentation strategies for I3D transfer learning and tests personalized per-subject modeling under temporally split segments.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Introduction

Dataset constraints and baseline performance

Sequential models and temporal granularity

Transfer learning and augmentation strategies

Experimental contributions

arXiv :2607 .07957v 1 [ cs .AI] 8 Jul 2026 Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies Raunak Mondal 1 and Peter Washington2 1 School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA 2 Department of Information and Computer Sciences, University of Hawai‘i at M¯anoa, Honolulu, HI, USA Abstract Autism spectrum disorder (ASD) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited. This study addresses two complementary challenges in automated detection of autism-related self-stimulatory behaviors from video: (1) identifying the optimal sequence-based neural network architecture and temporal sampling rate, and (2) characterizing data augmentation strategies for training on small behavioral datasets. For the first objective, long short-term memory (LSTM) and gated recurrent unit (GRU) models were trained on pose-derived features from the Self-Stimulatory Behavior Diagnosis (SSBD) dataset at frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames. Both architectures exceeded prior convolutional neural network (CNN) baselines (62–76% accuracy), with peak accuracies of 97.5%(LSTM) and 98.75%(GRU) at a sampling interval of every 15 frames. For the second objective, ten data augmentation strategies were applied to an I3D transfer learning pipeline, with an ablation study quantifying the marginal contribution of each technique. Horizontal flip achieved the highest standalone accuracy (48.78%), while exclusion of upsampling from the augmentation pipeline produced the largest performance degradation, indicating its necessity for augmentation applied to complex behavioral video. A personalized machine learning approach, in which per-subject models were trained and tested on temporally split segments of each video, produced consistent predictions (mean loss 1.84, SD 0.79) . These results provide practitioners with concrete guidance on architecture selection, sampling rate, and augmentation strategy for video-based behavioral classification in data-scarce clinical domains. 1 Introduction Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by differences in social communication and the presence of restricted, repetitive behaviors [1] . Current estimates place worldwide prevalence at approximately 75 million individuals, with reported rates in the United States increasing 241% since 2000 [2] . Despite this high prevalence, diagnosis remains labor-intensive: clinicians rely on structured behavioral observation instruments such as the Autism Diagnostic Observation Schedule (ADOS-2), which require trained administrators and multi-visit assessment pipelines [3] . These constraints create bottlenecks that delay diagnosis, particularly in low-resource settings where specialist access is limited. Automated recognition of self-stimulatory (“stimming”) behaviors from video offers one path toward scalable screening. Behaviors such as hand flapping, head banging, and spinning are among the motor stereotypies commonly observed in ASD and can in principle be detected from consumergrade video recordings [4] . However, computational approaches face two interrelated challenges. First, publicly available datasets of labeled stimming behaviors are small. The Self-Stimulatory Behavior Diagnosis (SSBD) dataset [4], one of the few purpose-built resources, contains only 75 videos across three behavior categories. Second, prior work has relied heavily on convolutional neural network (CNN) models, which achieve state-of-the-art accuracies of only 62–76% on these tasks [5] . Sequential architectures such as long short-term memory (LSTM) networks and gated recurrent units (GRUs) are well-suited to temporal pattern recognition and have shown strong performance in adjacent activity recognition domains [6, 7] . Yet their application to autism-s

FAQs

What main problems does the study address for video-based autism screening?

It tackles two challenges: selecting an optimal sequence-based model architecture with the right temporal sampling rate, and determining effective data augmentation strategies for training on small behavioral datasets.

Which models are evaluated, and what temporal sampling interval performs best?

The study trains LSTM and GRU models on pose-derived features and tests multiple frame sampling intervals. Peak performance occurs at a sampling interval of every 15 frames, reaching about 97.5% for LSTM and about 98.75% for GRU.

How do augmentation strategies affect performance in the I3D transfer learning pipeline?

Ten augmentation methods are tested with an ablation study to measure each technique’s marginal contribution. Horizontal flip gives the best standalone accuracy, while removing upsampling causes the largest performance drop, indicating upsampling is critical for complex behavioral video.

Jul 16, 2026

Upload by Genevieve