Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Quality

Jul 16, 2026

Upload by Miles

Underwater object detection is strongly affected by domain shift, where model performance varies across locations, habitats, and deployment conditions. Existing evaluations often rely on aggregate metrics that conceal failures in particular environments, while current benchmarks for domain generalization use synthetic changes that do not match real settings. A new framework assigns underwater domain labels based on appearance, scene composition, and acquisition geometry, enabling systematic study of both human annotation quality and detector performance. Results show substantial domain-dependent discrepancies and support interpretable measurement, benchmarking, and data/annotation planning for robust deployment.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Introduction

Problem: domain shift in underwater detection

Goal and contributions

Domain labeling framework

Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Quality Melanie Wille Dimity Miller Tobias Fischer Scarlett Raine QUT Centre for Robotics, Queensland University of Technology Brisbane, Australia {willemc, d24.miller, tobias.fischer, [sg.raine](sg.raine}@qut.edu.au)[}](sg.raine}@qut.edu.au)[@qut.edu.au](sg.raine}@qut.edu.au) arXiv :2607 . 10575v1 [ cs .CV] 12 Jul 2026 Abstract Underwater object detection is strongly affected by domain shift, where performance can vary significantly across different locations, habitats, and deployment conditions. However, detector performance is typically evaluated using aggregate metrics that hide failures in specific environments, while existing domain generalization benchmarks often rely on synthetic variations that do not reflect real-world conditions. We introduce a framework that characterizes underwater images by appearance, scene composition, and acquisition geometry to assign domain labels. Using this framework, we perform the first systematic study of how domain factors influence both human annotation quality in underwater object detection datasets and deep learning-based detector performance, revealing substantial domain-dependent discrepancies. By incorporating physically meaningful domain labels, domain shift becomes something we can characterize, measure, benchmark, and act on. We highlight how this can be used to guide data collection and annotation, design more informative benchmarks, and assess detector robustness across diverse underwater environments. 1. Introduction Underwater object detection is an important tool for marine science, enabling large-scale analysis of seafloor imagery to monitor benthic indicator species and assess ecosystem health under increasing human activity [6] . However, detector development often relies on collecting and annotating large amounts of training data, a process that is expensive and difficult to scale across new sites, habitats, and collection protocols [10, 31] . A major driver of data requirements is the large variation of underwater conditions, such as lighting, turbidity, depth, and environmental structure, which alter both image appearance and object characteristics, creating a wide range of distinct domains [24] . This often causes models trained on one set of conditions to perform Initial Human Labels Revised Ground Truth YOLO Predictions Holothurian – Echinus – Scallop – Starfish Figure 1 . Example underwater images from visually distinct underwater domains. Left: original RUOD labels. Middle: revised RUOD-R labels (considered closest to reality) . Right: YOLO26n predictions. Some true objects are missed (orange crosses ), while others were hallucinated (red crosses ) . Domain conditions challenge both humans and detectors. poorly when deployed in another, manifesting as domain shift, which challenges model generalization and limits data reusability [8, 10, 19] . Fig. 1 illustrates this challenge. Under changing underwater conditions, the same target classes may differ substantially in appearance and therefore detectability. In this paper, we investigate how such domain differences influence the performance of both object detectors and human annotators for consistently identifying and localizing targets inan image. Understanding the human side of the problem is particularly important because annotation quality directly determines the reliability of training and evaluation data, yet is typically assumed to be uniform across conditions. Existing underwater object detection benchmarks provide valuable resources for training and evaluating detectors [9, 11, 15, 22], but they are not designed to explain domaindependent failures. In this work, we use the term domain to refer to a subset of images sharing common environmental, scene, or acquisition conditions. Performance is typically summarized using a single dataset-level metric, such as mAP, which averages result

FAQs

What problem does the document address in underwater object detection?

It addresses domain shift, where detection performance changes significantly across different underwater locations, habitats, and acquisition conditions.

Why are existing benchmark evaluations insufficient?

They typically use aggregate dataset-level metrics (e.g., mAP) that hide failures specific to particular environments, and some domain generalization benchmarks rely on synthetic variations that do not reflect real deployment conditions.

How does the proposed framework characterize underwater domains?

It assigns interpretable domain labels using physical image properties, including appearance, scene composition, and acquisition geometry, to enable domain-aware analysis of both detector performance and annotation quality.

Jul 16, 2026

Upload by Miles