Orthogonal Dendritic Intrinsic Networks An Architecture for Significance-Ordered Orthogonal Latent Spaces

Jul 16, 2026

Upload by Asher

Principal Component Analysis (PCA)-like behavior—orthogonality and variance-based ordering of latent directions—is rarely achieved by deep autoencoders in a fully non-linear setting. ODIN (Orthogonal Dendritic Intrinsic Network) is introduced as an autoencoder architecture that recovers PCA-structured latent spaces by adding geometric constraints to the training objective. These constraints enforce mutually orthogonal latent dimensions ordered by explained variance, preserving neural expressiveness while improving interpretability. The approach is theoretically grounded and validated on synthetic and real-world datasets for interpretable structured feature learning and dimensionality reduction.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages43

Reading Time1 h 48 min

Introduction

Background

arXiv :2607 .05653v 1 [ cs .LG] 6 Jul 2026 Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces Jeanie Schreiber [jschrei@gmu.edu](jschrei@gmu.edu) Department of Mathematical Sciences George Mason University Fairfax, VA 22030, USA Tyrus Berry [tberry@gmu.edu](tberry@gmu.edu) Department of Mathematical Sciences George Mason University Fairfax, VA 22030, USA Zeeshan Ahmed [zeeshan.ahmed@nist.gov](zeeshan.ahmed@nist.gov) Sensor Science Division, Physical Measurement Laboratory NIST Gaithersburg, MD 20899, USA Editor: Abstract Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions tobe mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction. Keywords: Autoencoder, PCA, Non-linear PCA, Dimensionality Reduction, Feature Learning, interpretable ML 1 Introduction Traditional autoencoders consisting of an encoder and a decoder compress data into a latent space of lower dimensionality before the decoder attempts to reconstruct the original input from the latent representation. The loss function, usually based on reconstruction error, drives the model to learn efficient latent embeddings encoding important features of the data. While effective, the standard autoencoder architecture places no structural constraints on the latent space, often utilizing the entire latent representation for inference ©2026 Jeanie Schreiber, Tyrus Berry, and Zeeshan Ahmed. License: CC-BY 4.0, see [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/. Attribution)[. Attribution](https://creativecommons.org/licenses/by/4.0/. Attribution) requirements are provided at [http://jmlr.org/papers/v23/21-0000.html](http://jmlr.org/papers/v23/21-0000.html). Schreiber, Berry, and Ahmed and understanding. With no notion of ordering or separability, the resulting features are uniformly smeared across the latent space, making it challenging to interpret which latent dimensions correspond to meaningful input characteristics. The inherent ambiguity of latent space organization presents a common challenge in applying autoencoders for dimensionality reduction. The traditional autoencoder architecture produces latent representations with entangled, non-orthogonal directions that vary unpredictably across training runs. This instability complicates both reproducibility and interpretation. The Orthogonal Dendritic Intrinsic Network (ODIN) architecture addresses these limitations through a key innovation inspired by biology. First, ODIN employs a dendritic structure that enforces hierarchical importance ranking of latent dimensions. By restricting decoder access to cumulative subsets of latent variables (e.g., only the first k components), the network learns to prioritize directions that maximize reconstruction fidelity when progressively accrued. Additionally, strict latent space orthogonality is enforced through geometric constraints in the learning function, ensuring consistent axis alignment across training sessions. Together, these mechanisms enforce orthogonal and importance-ordered latent dimensions, mirroring PCA’s eigenvalue ranking while main

FAQs

What problem does ODIN address in standard deep autoencoders?

Standard autoencoders often produce latent representations without structural guarantees, leading to entangled, non-orthogonal directions and no consistent ordering by explained variance. This makes interpretation and reproducibility difficult across training runs.

How does ODIN enforce PCA-like structure in the latent space?

ODIN introduces a set of geometric constraints into the training objective to encourage strict orthogonality of latent dimensions. It also uses a dendritic structure that ranks latent dimensions by hierarchical importance through cumulative access during decoding.

Is ODIN supervised or unsupervised, and what benefits does it provide?

ODIN is trained in a completely unsupervised manner, without labeled data or predefined feature hierarchies. It enables stable ablation along variance-ranked axes and more reproducible latent assignments for cross-experiment comparisons.

Jul 16, 2026

Upload by Asher