Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding

Jul 16, 2026

Upload by Ophelia

Recent Multimodal Large Language Models (MLLMs) excel on single-view sports video understanding, but sports events often feature dense occlusion, rapid motion, and complex interactions that cannot be resolved from one camera. Existing benchmarks rarely test multi-view reasoning, even though real matches use multiple angles to provide complementary evidence for officiating. This work introduces SportMV-Bench with 787 multi-view video bundles and 2592 QA pairs across PAR, REI, and ADR, plus SportMV-Agent for iterative view selection, perception execution, and evidence-grounded reasoning, yielding 14.46% relative improvement.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

Introduction

SportMV-Bench benchmark design

Multi-view reasoning challenges in sports

Experiments and key findings

SportMV-Agent method overview

Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding Kerui Chen 1 ,∗ , Jinglu Wang2 , Xiaoyi Zhang2 , Yan Lu2 ,† 1Zhejiang University 2Microsoft Research Asia arXiv :2607 . 11844v1 [ cs .CV] 13 Jul 2026 Abstract Recent Multimodal Large Language Models (MLLMs) achieve strong performance on single-view video understanding benchmarks. However, sports videos involve dense occlusion, rapid motion, and complex interactions that are difficult to resolve from a single viewpoint. In practice, sports events are recorded from multiple camera angles, providing complementary evidence used by referees. Yet, no existing benchmark evaluates MLLMs on multi-view sports video understanding. To address this gap, we introduce SportMV-Bench, a comprehensive benchmark built from official match recordings, through a dedicated pipeline combining LLM-based generation, MLLM-based verification, and human filtering to ensure quality and consistency. SportMV-Bench containing 787 multi-view video bundlesand 2592 question-answer pairs across three categories: Perception-Aware Recognition (PAR), Rule-aware Event Interpretation (REI), and Adjudicative Decision Reasoning (ADR). Our analysis shows that current MLLMs fail to effectively exploit multi-view information, with the bottlenecks lying in fine-grained visual perception and view selection rather than logical reasoning or domain knowledge. We propose SportMV-Agent, an agentic framework that orchestrates an iterative loop of active view selection, perception tool execution, and evidence-grounded reasoning, achieving a significant 14.46% relative improvement over the strongest MLLM baseline. 1. Introduction Sports video understanding has received significant attention due to its wide applications in analytics [28], coaching [12], officiating support [15, 16], content retrieval [44], and audience engagement [29] . Recent Multimodal Large Language Models (MLLMs) [1, 2, 5, 21, 36] demonstrate strong capabilities in video understanding and reasoning, †Corresponding author. ∗ This work was done when K. Chen was an intern at Microsoft Research Asia. making them promising for sports video analysis. However, sports videos pose unique challenges for MLLMs, such as rapid movements, dense player interactions, domainspecific rules, and the need for fine-grained subtle visual cues. Several benchmarks have been developed to study sports video understanding, such as SoccerNet [11], SportsQA [22], and multi-sport datasets [41, 42] . While these benchmarks advance holistic understanding of broadcast videos, they typically assume that a single camera view provides sufficient evidence for question answering (QA) . In real-world, sports events are captured from multiple viewpoints to provide a complete perspective for refereesand audiences. Occlusion is common in sports: players often cluster around decisive moments, making critical actions hard to observe from a single view. Moreover, athletes may intentionally obscure events through exaggerated contact or simulated falls, making single-view evidence not only incomplete but sometimes misleading. As illustrated in Fig. 1(a), using only View 1 or View 2 leads to an incorrect out-of-bounds decision because the decisive touch is occluded, while incorporating View 3 reveals the correct outcome. Beyond occlusion, some queries require combining complementary evidence across views that may even appear conflicting. For example, in Fig. 1(b), View 1 localizes contact inside the penalty area, while View 2 shows that the defender did not touch the ball. Only by combining both views can the correct decision be made: a yellow card to attacker, and award opponents a defensive free kick rather than a penalty. These examples highlight the importance of multi-view reasoning for sports video understanding. Professional officiating systems such as VAR, Instant Replay, and Hawk-Eye already rely on multi-camera setups for this reason. However, e

FAQs

Why single-view sports video understanding is insufficient for MLLMs?

Sports videos involve dense occlusion, rapid motion, and subtle visual cues. Single-camera evidence can be incomplete or even misleading, requiring evidence aggregation across views.

What is SportMV-Bench and what does it contain?

SportMV-Bench is a multi-view sports video benchmark built from official match recordings. It includes 787 multi-view video bundles and 2592 QA pairs across perception-aware recognition, rule-aware event interpretation, and adjudicative decision reasoning.

How does SportMV-Agent improve performance over strong MLLM baselines?

SportMV-Agent orchestrates an iterative loop of active view selection, perception tool execution, and evidence-grounded reasoning. It achieves a 14.46% relative improvement over the strongest MLLM baseline by better exploiting decisive multi-view evidence.

Jul 16, 2026

Upload by Ophelia