VecHeart: Holistic Four-Chamber Cardiac Anatomy Modeling via Hybrid VecSets

Jul 16, 2026

VecHeart presents a unified framework for holistic reconstruction and generation of four-chamber cardiac anatomy, addressing the need to model intricate interrelations among heart structures under incomplete, sparse, or noisy observations. It replaces limitations of feed-forward implicit approaches by introducing a Hybrid Part Transformer with part-specific learnable queries and interleaved attention to capture inter-chamber dependencies. Anatomical Completion Masking and Modality Alignment further enable full-structure inference from missing chambers and heterogeneous imaging modalities. The framework extends to 3D+t dynamic mesh sequence generation and reports state-of-the-art high-fidelity results across challenging scenarios.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages11

Reading Time28 min

Introduction

Method

Hybrid Part Transformer

Anatomical Completion Masking

Modality Alignment

3D+t Mesh Sequence Generation

arXiv :2604 . 19403v2 [ cs .CV] 12 Jul 2026 VecHeart: Holistic Four-Chamber Cardiac Anatomy Modeling via Hybrid VecSets Yihong Chen 1 and Pascal Fua 1 1 CVLAB, EPFL {yihong.chen, [pascal.fua}@epfl.ch](pascal.fua}@epfl.ch) Abstract. Accurate cardiac anatomy modeling requires the model tobe able to handle intricate interrelations among structures. In this paper, we propose VecHeart, a unified framework for holistic reconstruction and generation of four-chamber cardiac structures. To overcome the limitations of current feed-forward implicit methods, specifically their restriction to single-object modeling and their neglect of inter-part correlations, we introduce Hybrid Part Transformer, which leverages partspecific learnable queries and interleaved attention to capture complex inter-chamber dependencies. Furthermore, we propose Anatomical Completion Masking and Modality Alignment strategies, enabling the model to infer complete four-chamber structures from partial, sparse, or noisy observations, even when certain anatomical parts are entirely missing. VecHeart also seamlessly extends to 3D+t dynamic mesh sequence generation, demonstrating exceptional versatility. Experiments show that our method achieves state-of-the-art performance, maintaining high-fidelity reconstruction across diverse challenging scenarios. Code is available at [https://github.com/Scalsol/VecHeart](https://github.com/Scalsol/VecHeart). Keywords: Cardiac Imaging · Shape Modeling · Generative Modeling 1 Introduction Accurate cardiac structure modeling is vital for applications like clinical diagnosis and intervention planning [27, 15 ,25] . However, faithfully representing multi-part heart geometry from diverse input configurations remains a significant challenge. Implicit functions offer significant flexibility and have been used across various fields. But conventional implicit-based shape representations [24,5 ,40 ,22] are typically tailored for single-structure modeling, neglecting the correlations among structures. Recent multi-part approaches [29,36 , 14 , 18] have begun to address this, but they rely on computationally expensive test-time optimization. Moreover, during test time, they require complete surface information to optimize the latent code. However, the observation is often sparse, and sometimes available only for some structures and not others. For instance, only sparse delineations of the four-chamber structure can be obtained from Long-Axis (LAX) views, while Short-Axis (SAX) MRI stacks cover the left and right ventricles, but not the atrial structures. These constraints severely limit their clinical utility. 2 Yihong Chen, Pascal Fua To address these limitations, we propose VecHeart, a versatile framework for high-quality four-chamber cardiac structure reconstruction and generation given potentially incomplete data. It keeps the fidelity of the individual parts while maintaining overall consistency across the whole heart. We start from the VecSet [39] latent representation to represent complex surfaces. Given surface points sampled from a 3D shape, VecSet uses a query vector and an encoder to turn them into a set of latent codes that are fed to a set of attention layers and then decoded into a signed distance function. VecHeart uses a similar encoder and decoder, along with a part-specific query vector. This enables us to introduce three new techniques designed to handle the heart’s complex multi-part anatomy even when the data is missing for some of them: – Hybrid Part Transformer. We sequentially and iteratively apply intrapart attention and inter-part attention to capture the correlations between the parts, as shown in Fig. 1(a) . – Anatomical Completion Masking. During training, we randomly mask some chambers and force VecHeart to still output the full anatomy from the incomplete data, as shown by Fig. 1(b) . This is key to handling missing data. – Modality Alignment. To allow reconstruction not only from

FAQs

What problem does VecHeart target in four-chamber cardiac modeling?

VecHeart targets accurate reconstruction and generation of four-chamber anatomy while preserving consistency among multiple structures, even when inputs are incomplete, sparse, or noisy.

How does Hybrid Part Transformer help model relationships between heart chambers?

It uses part-specific learnable queries and interleaved attention to capture correlations both within parts and across parts, improving inter-chamber dependency modeling.

How does VecHeart infer complete anatomy from missing or sparse observations?

It introduces Anatomical Completion Masking to train the model to output full anatomy despite masked chambers, and Modality Alignment to match representations from sparse CMR slices to denser samplings.

Jul 16, 2026

Upload by Olivia Brown