Revisiting Matching Response and Swept Feature Volumes for Wide-baseline Omnidirectional Stereo

Jul 16, 2026

Upload by Quinn

This paper presents a confidence-estimation training strategy for omnidirectional stereo under wide-baseline ambiguity, targeting frequent erroneous matches caused by global sweeping. By reinterpreting the expectation values of matching responses from a 3D encoder–decoder block as intrinsic confidence signals, the method penalizes ambiguous responses directly. It avoids auxiliary heads, multi-pass inference, and extra modules, improving efficiency and generalization. In addition, swept feature volume resampling uses regressed positive matching indices to enable joint prediction of meta-information such as surface normals, strengthening depth coherence through geometric regularization and contextual cues, with practical deployment for autonomous mobility.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Introduction

Confidence estimation for wide-baseline omnidirectional stereo

Limitations of entropy-based uncertainty under occlusions

Joint meta-information learning via swept feature volume resampling

Revisiting Matching Response and Swept Feature Volumes for Wide-baseline Omnidirectional Stereo Seungjin Jeon 1∗ Jongwoo Lim 1 ,2 and Changhee Won 1∗† arXiv :2607 . 11097v1 [ cs .CV] 13 Jul 2026 Abstract—In this paper, we propose a training strategy for confidence estimation in omnidirectional stereo, targeting the ambiguous matches that frequently occur in wide-baseline setups. Reinterpreting the matching responses produced by the 3D encoder–decoder block, we show that their expectation values provide intrinsic confidence signals. Building on this, our method directly penalizes ambiguous responses without auxiliary heads, multi-pass inference, or additional modules, resulting in more efficient and generalized predictions. Beyond confidence, we introduce swept feature volume resampling, where response features produced by 3D CNNs are resampled using regressed positive matching indices and then processed by 2D CNNs to predict meta-information such as surface normals. This joint learning introduces auxiliary geometric regularization and improves depth coherence by leveraging additional contextual cues during response aggregation stage. Experimental results demonstrate that our approach enhances both confidence estimation and surface normal prediction while maintaining deployment practicality for autonomous mobility applications. I. INTRODUCTION Omnidirectional perception has become an important component in various computer vision applications, particularly in robotics, autonomous driving, and aerial navigation, where accurate depth estimation critically affects system performance and safety. Although LiDAR sensors can be employed, recent advances in deep neural networks (DNNs) have greatly improved stereo matching [1], [2], making multi-camera omnidirectional stereo approaches increasingly attractive. Existing omnidirectional stereo methods typically involve configurations with multiple cameras [3]–[8] or stereo setups using 360◦ cameras [9]–[11] . More recently, approaches that employ wide-baseline cameras with global spherical sweeping have been proposed [3]–[7], [11] . In wide-baseline setups with global sweeping, occlusions without positive (valid) matches often lead to erroneous depth estimation results. For practical deployment on mobile platforms such as robots and drones, it is therefore essential to detect and handle such ambiguous matches using a confidence measure of the matching estimates. Various confidence estimation methods have been proposed for stereo matching. Poggi and Mattoccia [12], for example, feed the output disparity into separate 2D convolutional neural networks (CNNs) to predict confidence. Similarly, contextual features from input images [13], cost volumes [14], or iteratively ∗ Authors contributed equally. †Corresponding author. 1UVify Corporate Affiliated Research Institute, Republic of Korea.{seungjin.jeon, [changhee.won](changhee.won}@uvify.com)[}](changhee.won}@uvify.com)[@uvify.com](changhee.won}@uvify.com) [2](2 Seoul National University)[ Seoul National University](2 Seoul National University), [Seoul](Seoul), [Republic of Korea](Republic of Korea).{ [jongwoo.lim](jongwoo.lim}@snu.ac.kr)[}](jongwoo.lim}@snu.ac.kr)[@snu.ac.kr](jongwoo.lim}@snu.ac.kr) Fig. 1: From top: four wide field-of-view (FoV) fisheye input images and our multi-camera equipped drone, followed by inversedepth prediction, confidence prediction, and surface normal prediction from the proposed method. computed disparity profile [15] have been used as inputs to dedicated confidence estimation networks. Auxiliary heads within stereo matching networks have also been employed to predict confidence [16], [17] . However, such approaches typically incur higher inference time and architectural complexity, since they rely on additional network modules that go beyond the core stereo matching framework, limiting their practicality for real-time deployment. Without additional modules, Won et al. [3] use entropy as

FAQs

What problem does the paper address in wide-baseline omnidirectional stereo?

The approach performs swept feature volume resampling by resampling response features from 3D CNNs using regressed positive matching indices, then processing them with 2D CNNs. This enables joint learning of meta-information such as surface normals and adds geometric regularization for improved depth coherence.

Jul 16, 2026

Upload by Quinn