OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields

Jul 16, 2026

OmniX is a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion, enabling complete dynamic scene reconstruction across views and time. It disentangles dynamic foreground motion from static geometry, representing motion via a small set of dynamic tokens whose sparse, low-rank structure generates trajectory fields while preserving global interactions. An automatic UE5-based engine synthesizes 80K scenes and 1.28M multi-view videos with full annotations, achieving state-of-the-art dense 3D trajectory prediction and strong tracking performance, plus competitive depth and pose estimation.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Introduction

Related Work

Feedforward 3D and 4D Reconstruction

arXiv :2607 . 10840v1 [ cs .CV] 12 Jul 2026 OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields Yanqin Jiang 1 ,2†, Tengfei Wang3‡, Zhengwei Wang3 , Chenjie Cao3 , Junta Wu3 , Wenhan Luo4 , Weiming Hu 1 ,2 ,5 ,6 , Jin Gao 1 ,2 ,5‡, and Chunchao Guo3 1 MAIS, Institute of Automation, Chinese Academy of Sciences 2 School of Artificial Intelligence, University of Chinese Academy of Sciences 3 Tencent Hunyuan 4 HKUST 5 Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information 6 School of Information Science and Technology, ShanghaiTech University [jiangyanqin2021@ia.ac.cn](jiangyanqin2021@ia.ac.cn) ; [tengfeiwang12@gmail.com](tengfeiwang12@gmail.com) ; [jin.gao@nlpr.ia.ac.cn](jin.gao@nlpr.ia.ac.cn) [https://omnix4d.github.io/](https://omnix4d.github.io/) Abstract. Previous feed-forward 4D reconstruction methods either predict per-frame static point clouds, ignoring foreground motion, or estimate point cloud trajectories while being limited to small camera motions. This limits their ability to aggregate observations over time andreconstruct complete dynamic scenes under large viewpoint changes. To address this, we propose OmniX, a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion. 1) OmniX separates dynamic motion modeling from static geometry prediction and represents motion with a small set of dynamic tokens. Leveraging the sparse and low-rank structure of 3D motion, these tokens generate trajectory fields for all pixels in all images while efficiently preserving global interactions. 2) To facilitate training, we build an automatic UE5-based 4D data engine and introduce a dataset of 80k scenes and 1.28M multi-view videos with full geometric annotations. OmniX achieves state-of-the-art results on dense 3D point trajectory prediction and 3D point tracking, with competitive performance on video depth estimation and camera pose estimation. Keywords: 4D Reconstruction, 3D Point Tracking, Synthetic Dataset 1 Introduction Given image sequences of a dynamic scene, 4D reconstruction aims to reconstruct a dynamic 3D representation that is consistent across views and time. Serving as a foundation for dynamic scene understanding, it enables a wide range of downstream applications, including embodied AI simulation, autonomous driving perception, and AR/VR content creation. Thus, this area has attracted growing interest from the research community. † Work done during an internship at Tencent Hunyuan. ‡ Tengfei Wang and Jin Gao are co-corresponding authors. 2 Y. Jiang et al. Recent progress in 4D reconstruction has been largely driven by feed-forward approaches [5,7,18,32,38,41,46,48] . Most existing methods predict per-frame 3D point clouds from videos [7, 8, 16, 19, 38, 40, 46, 48], without explicitly modeling motion across time. To recover temporal correspondences, motion is often inferred by iteratively querying and tracking points across video frames [12,25,26, 41,42,44], which is computationally expensive and difficult to scale to dense predictions. A few works attempt to predict dense 3D point trajectories [5,18,32,33], but are typically limited to monocular videos with small camera motion. These methods struggle under large viewpoint changes because static geometry feature matching and dynamic foreground temporal correspondence learning are performed without disentanglement, thereby distracting from each other. In this work, we present OmniX, a 4D reconstruction framework that predicts dense 3D point trajectories from images captured at arbitrary viewpoints and times. It is robust to large camera motion and temporally discontinuous inputs by explicitly disentangling dynamic foreground motion from static scene structure. This is achieved by our proposed Sparse Spatiotemporal Attention (SSA) mechanism, which shows that, given the sparse and low-rank structure of 3D m

FAQs

What problem does OmniX target in feed-forward 4D reconstruction?

OmniX separates dynamic motion modeling from static geometry and represents motion with a small set of dynamic tokens, using their sparse and low-rank structure to generate per-pixel trajectory fields across views and time.

How does OmniX handle training data for large motion and wide multi-view coverage?

It builds an automatic UE5-based 4D data engine to synthesize dynamic scenes and render them with a customized camera system, producing a dataset of 80K scenes and 1.28M multi-view videos with depth, camera poses, and dense 3D point trajectory annotations.

What performance areas does OmniX achieve competitive or state-of-the-art results in?

OmniX reaches state-of-the-art results for dense 3D point trajectory prediction and 3D point tracking, and it also delivers competitive performance on video depth estimation and camera pose estimation.

Jul 16, 2026

Upload by Clementine