CoCoT-EEG Contrastive Pretrained Multiscale Convolutional Transformer for EEG Decoding

Jul 16, 2026

Upload by Aurelia

Self-supervised foundation models are promising for non-invasive electroencephalogram (EEG) decoding, yet common masked reconstruction pretraining can be ineffective for EEG due to high noise amplitude and information concentrated in limited frequency bands. CoCoT-EEG introduces contrastive pretraining with multiscale temporal convolution input layers and Transformer encoders. The method matches or exceeds reconstruction-pretrained EEG models on heterogeneous electrode decoding benchmarks, and training from scratch also improves single-task baselines. Ablations validate contrastive learning feasibility and highlight key architectural choices for future EEG FM research.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

Abstract

Introduction

Motivation and limitations of reconstruction pretraining

Contrastive learning for robust EEG representations

Main contributions

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding Gabriel Mahuas∗ Sigma Nova Paris, France [gabriel.mahuas@sigmanova.ai](gabriel.mahuas@sigmanova.ai) Victoria Shevchenko∗ Sigma Nova Paris, France [victoria.shevchenko@sigmanova.ai](victoria.shevchenko@sigmanova.ai) Ugo Tanielian∗ Sigma Nova Paris, France [ugo.tanielian@sigmanova.ai](ugo.tanielian@sigmanova.ai) Yassir Bendou Sigma Nova Paris, France [yassir.bendou@sigmanova.ai](yassir.bendou@sigmanova.ai) arXiv :2607 .09543v 1 [ cs .LG] 10 Jul 2026 Richard Gao Goethe University Frankfurt Frankfurt, Germany [r.dg.gao@gmail.com](r.dg.gao@gmail.com) Abstract Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data—like EEG—with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT) . CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture’s flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies. 1 Introduction Electroencephalography (EEG) is pervasive in both research and clinical practice thanks to its moderate acquisition costs, non-invasive nature, and high temporal resolution. Decoding from EEG data remains an important challenge which, when addressed, enables myriad applications such as brain-computer interface (BCI) and clinical diagnosis. Classical deep learning-based solutions use deep neural networks in a supervised setting while extracting information from the time series, e.g., using temporal convolutions as the inductive bias of choice [1, 2, 3, 4] . While such solutions are generally effective, they are single-task models and often do not generalize well across settings [5] . ∗ Equal contribution. Preprint. Recently, large-scale self-supervised pretraining has proven successful at boosting cross-domain generalizability in language and computer vision. By leveraging vast amounts of unlabeled data, these models learn low-dimensional representations of their inputs that serve as generically useful features for downstream applications, often requiring little to no finetuning. Because of their general-purpose nature, such models have come to be known as foundation models (FMs) [6] . This pretrained FM strategy has also been adopted for EEG decoding: recent models like LaBraM [7], CBraMod [8], CSBrain [5], LUNA [9], and REVE [10] are pretrained on tens of thousands of hours of EEG data and perform well on a diverse set of downstream decoding tasks. Inspired by vision and language models, these solutions have converged on a particular recipe of early patch tokenization of the input data with reconstruction-based pretraining objectives (e.g., masked autoencoding, MAE [11]) . By further adapting this recipe to EEG data, such as discretizing into a codebook (via VQ-VAE [7]), dedicated spatial/temporal attention [8, 5], and more sophisticated electrode position encoding strategies [10, 5] combined with increasingly massive pretraining datasets [10], pretrained EEG FMshave emerged as potential alte

FAQs

What problem does the document address in EEG foundation model pretraining?

The document targets limitations of reconstruction-based pretraining for EEG, especially in high-noise settings where information is concentrated in narrow frequency bands and early tokenization/reconstruction can be suboptimal.

What is CoCoT-EEG and how does it differ from prior reconstruction-pretrained approaches?

CoCoT-EEG uses a convolution-first design with multiscale 1D convolutions followed by Transformer encoder blocks, and it applies contrastive pretraining objectives rather than masked reconstruction.

How does CoCoT-EEG perform compared with existing single-task and pretrained EEG models?

The model trained from scratch outperforms previous best single-task approaches and can rival pretrained models. With contrastive pretraining, it further improves results across diverse downstream tasks, outperforming recent reconstruction-pretrained state of the art on the reported benchmarks.

Jul 16, 2026

Upload by Aurelia