Prompting-MammAlps Fine-Grained Text-to-Video Retrieval for Camera-Trap Data

Jul 16, 2026

Upload by Maeve

Automatically retrieving videos from large camera-trap datasets remains challenging in ecological analysis. Prompting-MammAlps presents a new camera-trap TVR benchmark and a fine-grained, interpretable text-to-video retrieval method. A vision transformer performs spatiotemporal action localization and converts results into structured text per video. An LLM-based coding agent parses this representation using function calls to reduce hallucinations. The method reaches 34% set-based F1 on 135 queries over 775 candidates, outperforming a best zero-shot VLM at 18%.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages28

Reading Time1 h 11 min

Introduction

Wildlife monitoring and camera-trap data

Challenges in fine-grained ecological video processing

Text-to-Video retrieval with video-language models

Prompting-MammAlps contribution

Method Overview

Spatiotemporal action localization

Structured text representation

LLM-based parsing and retrieval with function calls

Benchmark and Evaluation

Query set and candidate videos

Performance comparison with zero-shot VLM

arXiv :2607 .09876v1 [ cs .CV] 10 Jul 2026 Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data Valentin Gabeff1 , Baptiste Maquignaz 1 , Jennifer Shan 1 , Sepideh Mamooler 1 , Gencer Sumbul 1 , Blair Costelloe2 ,3 , Devis Tuia 1 , and Alexander Mathis 1 ,4 1 Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland 2 Max Planck Institute of Animal Behavior, Konstanz, Germany 3 University of Konstanz, Konstanz, Germany 4 [alexander.mathis@epfl.ch](alexander.mathis@epfl.ch) Abstract. Automatically retrieving videos from large camera-trap datasets remains challenging. Text-to-Video retrieval (TVR) methods based on large video-language models (VLMs) have potential to retrieve events of interest by describing them with simple text queries. However, current methods often lack spatiotemporal understanding and do not generalize well to ecological data. In this work, we introduce Prompting-MammAlps, the first camera-trap TVR benchmark, and propose a fine-grained and interpretable TVR method. Specifically, we trained a vision transformer to performspatiotemporal action localization, and convert its output to structured text, describing each video. Independently, ethology-inspired queries are processed by a Large-Language Model (LLM) based coding agent to parse the structured text per video and retrieve videos accordingly. We harnessed the LLM to use functions from a custom parsing library to minimize the risk of LLM hallucinations and to improve method interpretability. This retrieval approach applied on the Prompting-MammAlps benchmark achieved a set-based F1-score of 34% on a test set of 135 ecologically-relevant queries and 775 candidate videos. In comparison the best zero-shot VLM achieved a F1-score of 18%, while also lacking interpretability. [Project page](Project page: cnai.epfl.ch/prompting-mammalps)[: cnai.epfl.ch/prompting-mammalps](Project page: cnai.epfl.ch/prompting-mammalps) Keywords: Text-to-Video Retrieval · Camera-Traps · Benchmark - 135 text queries | "An adult female red deer nursing ." | | --- | | "A video showing individuals from at least two different species ." | | "A roe deer foraging in rainy weather ." | . . . Videos matching the query (N=51) Not matching the query (N=2814) Densely annotated for up to 6 attributes a) Prompting-MammAlps Benchmark Fig. 1: Overview of Prompting-MammAlps and our TVR method. (a) We associate a densely annotated pool of camera-trap videos with 135 queries spanning six fine-grained visual attributes and diverse spatiotemporal relationships. (b) We perform spatiotemporal action localization on the candidate videos and store each result ina structured .json format. For each query, we task a LLM to write a parsing function that assesses if the text representation of a candidate video matches the query or not. 2 V. Gabeff et al. 1 Introduction Wildlife monitoring is crucial to design accurate ecosystem conservation strategies, to measure their impact over time [21, 39, 69, 73], and more generally to improve our fundamental understanding of how animals behave in complex, real-world environments [2, 8, 20, 42] . To achieve this, ecologists deploy networks of camera-traps –among other remote sensors and other techniques– that capture fine-grained visual information of ecological events with minimal disturbance [11, 13, 57, 66, 67] . However, processing this data remains time-consuming, far exceeding the time spent in data analysis [5, 73, 88] . While deep learning advances have been improving camera-trap data processing, for example to filter out false positive images [6, 32] or to recognize species identity for some taxonomic groups [61]; fine-grained visual attributes (e.g. animal behavior) are still typically annotated manually by experts [34, 66, 88] or through citizen-science initiatives with notable exceptions [29, 49, 56, 62] . When recording videos, camera-traps can capture rich information on a

FAQs

What problem does Prompting-MammAlps address?

It targets automatic text-to-video retrieval for large camera-trap datasets, where existing approaches struggle with spatiotemporal understanding and generalization to ecological data.

How does the proposed TVR method work end to end?

A vision transformer localizes spatiotemporal actions and outputs structured text for each video; an LLM-based coding agent then parses that structured text to decide whether candidates match each query.

How does the method reduce risks of hallucination and improve interpretability?

It uses the LLM to call functions from a custom parsing library, constraining how structured text is interpreted and making retrieval decisions more interpretable.

Jul 16, 2026

Upload by Maeve