4DR360° State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception

Jul 16, 2026

Upload by Ophelia

Reliable autonomous driving depends on full-scene perception that unifies foreground object understanding with dense semantic scene layout. While 4D millimeter-wave radar is robust and affordable, its sparse returns necessitate radar-camera fusion for comprehensive perception. This work introduces 4DR360°, a 4D radar-camera framework that treats semantic occupancy as a persistent scene state, enabling cross-modal state reasoning. SBE strengthens intra-frame BEV features and DTF propagates state evidence temporally. Experiments extend ManTruckScenes and OmniHD-Scenes with unified joint detection-and-occupancy evaluation, reporting accuracy, robustness, ablations, and efficiency; code and labels are released after acceptance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

Abstract

Introduction

Related Work and Motivation

Framework Overview

Cross-modal state reasoning with SBE and DTF

Cross-dataset joint protocol (ManTruckScenes and OmniHD-Scenes)

4DR360◦ : State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception Xiaokai Bai 1 , Lianqing Zheng2 , Runwei Guan3 , Songkai Wang 1 , Siyuan Cao 1 , Hui-liang Shen 1 1 College of Information Science and Electronic Engineering, Zhejiang University. 2 School of Automotive Studies, Tongji University. 3Thrust of Artificial Intelligence, Hong Kong University of Science and Technology. [shawnnnkb@gmail.com](shawnnnkb@gmail.com) arXiv :2607 .09629v1 [ cs .CV] 10 Jul 2026 Abstract Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout. Recently, 4D millimeter-wave radar has emerged as a robust and affordable sensor, yet its sparse returns make radar-camera fusion necessary for comprehensive scene understanding. Existing radar-camera methods mainly optimize detection, while dual-task systems usually decode boxes and occupancy with limited interaction. To address this gap and advance radar-based multi-task learning, we propose 4DR360◦ , a 4D radar-camera framework for 360◦ full-scene perception, which models semantic occupancy as a persistent scene state rather than a terminal output. 4DR360◦ follows a cross-modal state reasoning paradigm, where the occupancy state is modeled and propagated through stages for coarse-to-fine feature aggregation. Specifically, State-guided BEV Enhancement (SBE) strengthens intra-frame BEV representation, while Doppler-guided Temporal Fusion (DTF) preserves state evidence over longer temporal horizons. Beyond the model, we further extend ManTruckScenes with satellite-map-based generated occupancy labels and pair it with OmniHD-Scenes in a unified cross-dataset detection-and-occupancy protocol. The resulting experiments cover accuracy, robustness, ablation, and efficiency under one radar-camera multi-task evaluation framework. Code and labels will be released upon acceptance. Introduction Autonomous driving requires 360◦ full-scene perception that jointly models foreground agents and surrounding scene layout. In this setting, 3D object detection estimates compact boxes for traffic participants (Liu et al. 2023; Bai et al. 2024, 2026c; Xia et al. 2026), whereas semantic occupancy prediction recovers dense free, occupied, and semantic voxel states around the ego vehicle (Cao and De Charette 2022; Li et al. 2023; Wei et al. 2023; Tian et al. 2023; Zheng et al. 2024) . These two tasks describe the same scene at different granularities. Detection focuses on instance-level localization, while occupancy supplies the spatial support and layout context in which those instances exist. Therefore, reliable full-scene perception requires a unified treatment of object reasoning and scene layout. Driven by recent datasets and radar-camera perception methods, 4D millimeter-wave radar is becoming an Copyright © 2026 . All rights reserved. (d) 4DR360° (Ours) Cross-modal State Reasoning Multi-view Joint Task Figure 1 . Comparison of representative 4D radar-camera paradigms. (a) SGDet3D is front-view and detection-only,(b) RCBEVDet extends detection to 360◦ perception,(c) Doracamom supports joint detection and occupancy prediction without explicit state reasoning, and (d) 4DR360◦ organizes 360◦ radar-camera multi-task perception through occupancy-state reasoning with SBE and DTF. important sensing modality for autonomous driving (Palffyet al. 2022; Zheng et al. 2022; Paek, Kong, and Wijaya 2022; Zheng et al. 2023; Lin et al. 2024; Bai et al. 2024, 2026c; Xia et al. 2026) . It provides stable geometric and motion cues under challenging conditions. However, its sparse returns still require camera semantics for complete scene understanding. Recent radar-camera methods therefore improve bird’s-eye-view fusion, semantic-geometric interaction, sparse object representation, and temporal modeling (Zheng et al. 2023; Lin et al. 2024; Bai et al. 2024, 2026c; Xia et al. 2026) . Nevertheless, most o

FAQs

What problem does 4DR360° address in radar-camera full-scene perception?

It targets the gap where radar-camera fusion systems mainly optimize detection, while occupancy and detection are not sufficiently coupled through interactive, state-aware reasoning for full-scene understanding.

How does 4DR360° model occupancy in the proposed framework?

Occupancy is modeled as a persistent scene state rather than a terminal output, and the state is represented and propagated through stages for coarse-to-fine aggregation.

What roles do SBE and DTF play in the method?

State-guided BEV Enhancement (SBE) strengthens intra-frame BEV representation, while Doppler-guided Temporal Fusion (DTF) preserves state evidence over longer temporal horizons.

Jul 16, 2026

Upload by Ophelia