RABBiT: Rapidly Adaptive BOLD Foundation Model via Brain-Tuning for Accurate Zero-Shot and Few-Shot Speech-Evoked fMRI Prediction

Jul 16, 2026

Upload by Liam

Language understanding in the brain depends on experimental context and varies strongly across individuals, making cross-subject generalization difficult. RABBiT introduces a compact audio-to-fMRI foundation model that captures shared structure while rapidly adapting to new participants and inputs. Evaluated on 324 participants across multiple unseen fMRI datasets, it delivers accurate zero-shot prediction of speech-evoked responses in auditory and language-selective regions and surpasses prior approaches. With about 10 minutes of subject-specific calibration data, parameter-efficient tuning further improves few-shot performance and enables interpretability via learned region-specific attention and shared–idiosyncratic decomposition.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages26

Reading Time1 h 6 min

Abstract

Introduction

Overview of RABBiT

Cross-attention readout

Training pipeline

Method: Brain-Tuning, Temporal Brain Transformer, SID Decomposition

RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain arXiv :2607 .05 17 1v 1 [ cs .CL] 6 Jul 2026 Omer Moussa Max Planck Institute for Software Systems Saarbrücken, Germany [omoussa@mpi-sws.org](omoussa@mpi-sws.org) Mariya Toneva Max Planck Institute for Software Systems Saarbrücken, Germany [mtoneva@mpi-sws.org](mtoneva@mpi-sws.org) Abstract Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generalize across both. This calls for a foundation model of languageevoked brain activity that can capture shared structure while adapting efficiently to new participants and inputs. We introduce RABBiT (Rapidly Adaptive BOLD foundation model via BraIn-Tuning), a compact audio-to-fMRI encoder designed for accurate zero-and few-shot prediction. A comprehensive evaluation on 324 participants across multiple unseen fMRI datasets shows that RABBiT enables accurate zero-shot prediction of fMRI responses to natural speech across auditory and language-selective regions, surpassing the SOTA foundation model for fMRI and predictions based on group averages. With as little as 10 minutes of participantspecific data, RABBiT further improves performance via parameter-efficient tuning, substantially outperforming per-participant linear models. RABBiT’s performance is driven by two key innovations: (1) learned region-specific attention, and (2) a decomposition of brain responses into shared and subject-specific components, combined with a brain-tuned speech backbone. In addition to supporting strong predictive accuracy, the structured, region-specific representations that RABBiT learns enable interpretability. By eliminating the need for extensive per-participant data and model fitting, RABBiT enables scalable population-level analyses of language in the human brain. We make the code available at [https://github.com/bridge-ai](https://github.com/bridge-ai)neuro/rabbit. 1 Introduction Natural language comprehension evokes brain responses that are both reliable across listeners and strongly shaped by individual variation. Early auditory regions exhibit relatively consistent responses across participants, whereas higher-order language regions vary substantially across brains [24] . This creates a central challenge for language fMRI modeling: a useful encoder must capture the shared structure of language processing while remaining adaptable to individual listeners. Most existing brain encoding approaches address only one side of this problem. Standard voxelwise encoding models fit a separate readout for each participant [16, 2] . Although effective with sufficient data, these methods require collecting new fMRI recordings and fitting a high-dimensional model for every participant. More recent brain-tuning approaches improve the underlying speech representations using fMRI supervision [21, 23, 31], but still rely on participant-specific voxelwise fitting at inference time. In parallel, recent foundation-style brain encoders improve zeroshot prediction across participants [7, 5], yet they are not designed for efficient adaptation to new Preprint. (c) Training: predict fMRI per ROI, compare to measured. Figure 1: Overview of RABBiT. (a) A single encoder that supports two transfer regimes for new subjects. Zero-shot: predicting group-level responses; tested on 324 unseen participants across 16 held-out studies with naturalistic listening; RABBiT’s predictions on auditory ROIs already saturate near the inter-subject consistency. Few-shot: adapting to a new subject with small data; RABBiT rapidly and effectively adapts to unseen subjects with only ∼ 10 minutes of paired audio, improving idiosyncratic language ROIs with 3 orders of magnitude fewer parameters than voxel-wise models.(b) The cross-attention readout: learnabl

FAQs

What problem does RABBiT address in language fMRI modeling?

It addresses the need to model language-evoked brain activity that is both shared across listeners and adaptable to individual variation, which is difficult for existing encoders to generalize across participants.

How does RABBiT support zero-shot versus few-shot prediction?

Zero-shot predicts group-level fMRI responses for unseen subjects. Few-shot adapts to a new participant using a small amount of paired audio data (around 10 minutes) with parameter-efficient tuning to improve subject-specific regions.

What design components drive RABBiT’s performance?

RABBiT uses learned region-specific attention through a Temporal Brain Transformer, and a Shared–Idiosyncratic Decomposition that splits predicted responses into shared and subject-specific components, combined with a brain-tuned speech backbone.

Why is RABBiT considered more scalable than per-participant fitting approaches?

It reduces or eliminates the need for extensive per-participant data collection and heavy voxel-wise model fitting, enabling scalable population-level analyses while maintaining strong predictive accuracy.

Jul 16, 2026

Upload by Liam