Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

Jul 16, 2026

In-context learning (ICL) can be realized as implicit gradient descent embedded in a model’s forward pass, but reproducing this capability in biologically plausible spiking neural networks (SNNs) remains unresolved. Prior SNNs fail the Garg-2022 benchmark due to a structural assumption: adaptation is routed through inference-time synaptic plasticity, with dendrites treated as passive conduits. This work introduces DendriCL, where a single dendritic compartment’s subthreshold dynamics implement an online learning rule structurally equivalent to leaky Widrow–Hoff LMS, achieving seed stability on challenging tasks and yielding a linear-probe recovery of the reference trajectory (R2 = 0.93).

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages26

Reading Time1 h 6 min

Introduction

Background: ICL as implicit gradient descent

Gap: ICL in spiking neural networks

Key claim and structural reinterpretation

Dendritic In-Context Learning in a Single-Layer Spiking Neural Network arXiv :2607 .02283v 1 [ cs .NE] 2 Jul 2026 Juwei Shen Department of Computing, FCMS The Hong Kong Polytechnic University Hong Kong, China [sheldon.shen@connect.polyu.hk](sheldon.shen@connect.polyu.hk) Yujie Wu Department of Computing, FCMS The Hong Kong Polytechnic University Hong Kong, China [yu-jie.wu@polyu.edu.hk](yu-jie.wu@polyu.edu.hk) Changwen Chen∗ Department of Computing, FCMS The Hong Kong Polytechnic University Hong Kong, China [changwen.chen@polyu.edu.hk](changwen.chen@polyu.edu.hk) Abstract In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures—Transformers [Akyürek et al., 2023, von Oswald et al., 2023], Mamba [Park et al., 2024], state-space models [Sushma et al., 2024], and MLPs [Tong and Pehlevan, 2025] . Capturing this capability in biologically plausible Spiking Neural Networks (SNNs) has remained an open challenge: existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions. We trace this failure to a structural assumption: prior SNN designs route adaptation through inference-time synaptic plasticity, viewing the dendritic compartment as a passive conduit for error or teacher signals. We challenge this assumption. The subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm. By treating the compartment as the computational substrate rather than a passive conduit, we propose DendriCL—a single-layer compartmental spiking architecture whose apical recurrence is structurally identical to leaky online Widrow–Hoff LMS [Widrow and Hoff, 1960] . This dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer. DendriCL is uniquely seed-stable at super-dimensional Garg-2022 ICL—where dense Transformers exhibit grokking-style instability and fail past moderate task dimension—and a linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R2 = 0 .93, showing the algorithm is structurally embedded in the dynamics rather than implicitly discovered during training. Taken together, ICL requires neither attention, depth, nor inference-time plasticity: a single compartment with online-LMS dynamics is sufficient. 1 Introduction In-context learning (ICL)—the ability of a trained sequence model to solve a new task from a handful of labeled examples in its prompt, without any parameter update [Brown et al., 2020]—has been mechanistically traced, across architecture families, to an implicit gradient-descent algorithm embedded in the trained forward pass. On the synthetic function-class protocol of Garg et al. [2022], ∗ Corresponding Author Preprint. Figure 1: DendriCL overview. (A) Biological layer-5 cortical pyramidal neuron: apical tuft receivestop-down feedback, basal dendrites receive bottom-up input, and the soma integrates both. (B) DendriCL computational model—a single compartmental layer implementing the apical-basal-soma architecture with a learned online-LMS update in the apical membrane potential uA . Synaptic weights WA , WB , Wout are frozen at inference; the apical state is not reset by spikes and evolves across the full context. (C) Garg 2022 ICL task: given k labeled pairs (xi , yi) the model predicts yˆq for a query xq . The apical membrane trajectory converges to the task parameter w with linear-probe decoding R2 = 0 .93 into the reference online-LMS estimate. Akyürek et al. [2023] showed that trained Transformers internally recover the iterates of a gradientbased learner; von Oswald et al. [2023] made this equivalence explicit (linear self-attention ≡ one GD step on linear regression; stacking ≡ trajectory); Ahn et al. [2024] sharpened it topreconditioned GD. The reading has since generalized well beyond attention: state-space models [Sushma et al., 2024], Mamba [Park et al., 2024, Grazz

FAQs

What prevents most spiking neural networks from achieving general-purpose in-context learning on the Garg-2022 protocol?

The document attributes failure to a structural assumption that routes adaptation through inference-time synaptic plasticity while treating the dendritic compartment as a passive conduit for teacher or error signals.

How does DendriCL enable in-context learning without inference-time plasticity?

DendriCL freezes synaptic weights at inference and uses the subthreshold dynamics of a single dendritic compartment to implement a complete online learning algorithm structurally equivalent to leaky Widrow–Hoff LMS via apical recurrence.

What evidence shows that the learned algorithm is embedded in dendritic dynamics?

A linear probe decodes the apical membrane trajectory to recover the reference online-LMS estimate directly, achieving R2 = 0.93, indicating the dynamics contain the algorithm rather than discovering it implicitly during training.

Jul 16, 2026

Upload by Ava Thompson