Machine learning to predict killer whale (Orcinus orca) behaviors using partially labeled vocalization data

Aug 4, 2026

Upload by Eden

Machine learning models infer killer whale behavior from recorded vocalizations, using sound as a proxy for otherwise difficult-to-collect behavioral observations. The paper addresses the key obstacle of limited labeled data, where recording-level behavior labels can over-include irrelevant behaviors for individual sound segments. A ResNet-34 architecture plus a custom partially labeled learning loss achieves 96.1% general behavior classification accuracy on previously unseen segments.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Calls and vocal repertoires in killer whales

Motivation: links between vocalizations and behavior

Study goal and partially labeled learning challenge

TYPE Original Research PUBLISHED 13 June 2025 DOI 10.3389/fmars.2025.1232022 OPEN ACCESS EDITED BY Clea Parcerisas, Flanders Marine Institute, Belgium REVIEWED BY Brigitte Schlögl, Leipzig University, Germany Maxence Ferrari, UPR7051 Laboratoire de me´canique etd’acoustique (LMA), France *CORRESPONDENCE Sophia Sandholm [ssandhol@andrew.cmu.edu](ssandhol@andrew.cmu.edu) RECEIVED 31 May 2023 ACCEPTED 20 May 2025 PUBLISHED 13 June 2025 CITATION Sandholm S (2025) Machine learning to predict killer whale (Orcinus orca) behaviors using partially labeled vocalization data. Front. Mar. Sci. 12:1232022 . doi: 10.3389/fmars.2025.1232022 COPYRIGHT © 2025 Sandholm. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Machine learning to predict killer whale (Orcinus orca) behaviors using partially labeled vocalization data Sophia Sandholm* Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, United States Orcinus orca (killer whales) exhibit complex calls. In a call, an orca typically varies the frequencies, varies the length, varies the temporal patterns, varies their volumes, and can use multiple frequencies simultaneously. Behavior data is hard to obtain because orcas live under water and travel quickly. Sound data is relatively easy to capture. This paper studies whether machine learning can predict behavior from vocalizations. Such prediction would help scientiﬁc research and have safety applications because one would like to predict behavior while only having to capture sound. A signiﬁcant challenge in this process is lack of labeled data. This paper works with recent recordings of McMurdo Sound orcas where each recording is labeled with the behaviors observed during the recording. This yields a dataset where sound segments—continuous vocalizations that can be thought of as call sequences or more general structures—within the recordings are labeled with potentially superﬂuous behaviors. This is because in a given segment, an orca may not be exhibiting all of the behaviors that were observed during the recording from which the segment was taken. Despite that, with a careful combination of recent machine learning techniques, including a ResNet-34 convolutional neural network and a custom loss function designed for partially labeled learning, a 96.1% general behavior label classiﬁcation accuracy on previously unheard segments is achieved. This is promising for future research on orca behavior as well as language and safety applications. KEYWORDS orca, vocalization, calls, behavior prediction, machine learning, partially labeled learning, language, semantics 1 Introduction Marine biologists have recordings of Orcinus orca (killer whale) vocalizations in which they have identiﬁed what they coin “calls” [e.g., Poupard et al. (2021)] . Generally speaking, calls fall into two categories, whistles or pulsed calls, and are used for communication (Ford, 1989, 1991) . These calls are variable in length and can be classiﬁed into discrete call Frontiers in Marine Science 01 [frontiersin.org](frontiersin.org) types according to temporal patterns, fundamental frequency, and duration, among other features (Wellard et al., 2020a; Ford, 1989) . Even within an individual call, an orca varies the frequencies, varies their volumes, and can use multiple frequencies simultaneously, as in the case ofbiphonic calls (Filatova, 2020; Ford, 1989).In addition, orcas produce echolocation clicks which they use to observe their surroundings (Ford, 1989; Schevill and Watkins, 1966; Leu et al.,

FAQs

Why is behavior prediction from killer whale vocalizations important?

Because behavior observations are hard to obtain underwater and sound is easier to capture, enabling both scientific research and potential safety applications.

What makes the labeling problem difficult in this dataset?

Recordings are labeled with behaviors observed during the recording, but individual sound segments may not contain all behaviors present in the full recording, creating potentially superfluous labels.

Which model and training approach are used to handle partially labeled data?

The approach combines a ResNet-34 convolutional neural network with a custom loss function designed for partially labeled learning.

Aug 4, 2026

Upload by Eden