Demystifying image-based machine learning: a practical guide to automated analysis of field imagery using modern machine learning tools

Aug 2, 2026

Upload by Patrick

Image-based machine learning methods are increasingly used for data analysis across science and industry because they can rapidly and automatically extract contextual and spatial information from images, reducing reliance on large human labor. Despite strong demonstrated potential in ocean research, tools remain underused in species and environmental monitoring, biodiversity surveys, fisheries estimation, rare-event detection, animal behavior studies, and citizen science. This review provides an approachable end-to-end guide for applying image-based machine learning effectively, including data preparation, model training and deployment, and diagnosing issues that cause poor performance on new imagery.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

AccessFree to download

Introduction

Challenges of visual data and high-dimensional imagery

Role of image analysis and objective of image-based machine learning

TYPE Review PUBLISHED 05 June 2023 DOI 10.3389/fmars.2023.1157370 OPEN ACCESS EDITED BY Mark C. Benﬁeld, Louisiana State University, United States REVIEWED BY Nils Piechaud, Norwegian Institute of Marine Research (IMR), Norway Lina Zhou, Hong Kong Polytechnic University, Hong Kong SAR, China *CORRESPONDENCE Andrew M. Hein [andrew.hein@cornell.edu](andrew.hein@cornell.edu) RECEIVED 08 February 2023 ACCEPTED 19 May 2023 PUBLISHED 05 June 2023 CITATION Belcher BT, Bower EH, Burford B, Celis MR, Fahimipour AK, Guevara IL, Katija K, Khokhar Z, Manjunath A, Nelson S, Olivetti S, Orenstein E, Saleh MH, Vaca B, Valladares S, Hein SA and Hein AM (2023) Demystifying image-based machine learning: a practical guide to automated analysis of ﬁeld imagery using modern machine learning tools. Front. Mar. Sci. 10:1157370 . doi: 10.3389/fmars.2023.1157370 COPYRIGHT © 2023 Belcher, Bower, Burford, Celis, Fahimipour, Guevara, Katija, Khokhar, Manjunath, Nelson, Olivetti, Orenstein, Saleh, Vaca, Valladares, Hein and Hein. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Demystifying image-based machine learning: a practical guide to automated analysis of ﬁeld imagery using modern machine learning tools Byron T. Belcher 1, Eliana H. Bower 1, Benjamin Burford 1,2, Maria Rosa Celis 1,2, Ashkaan K. Fahimipour 1,2,3, Isabela L. Guevara 1, Kakani Katija 4, Zulekha Khokhar 1, Anjana Manjunath 1, Samuel Nelson 1,2, Simone Olivetti 1,2, Eric Orenstein 4, Mohamad H. Saleh 1, Brayan Vaca 1, Salma Valladares 1, Stella A. Hein 1,5 and Andrew M. Hein 1,6* 1AI for the Ocean program, University of California Santa Cruz, Santa Cruz, CA, United States, 2 Institute of Marine Sciences, University of California Santa Cruz, Santa Cruz, CA, United States, 3 Florida Atlantic University, Department of Biology, Boca Raton, FL, United States, 4 Monterey Bay Aquarium Research Institute, Research and Development, Moss Landing, CA, United States, 5Cornell University, College of Agriculture and Life Sciences, Ithaca, NY, United States, 6 Department of Computational Biology, Cornell University, Ithaca, NY, United States Image-based machine learning methods are becoming among the most widelyused forms of data analysis across science, technology, engineering, and industry. These methods are powerful because they can rapidly and automatically extract rich contextual and spatial information from images, a process that has historically required a large amount of human labor. A wide range of recent scientiﬁc applications have demonstrated the potential of these methods to change how researchers study the ocean. However, despite their promise, machine learning tools are still under-exploited in many domains including species and environmental monitoring, biodiversity surveys, ﬁsheries abundance and size estimation, rare event and species detection, the study of animal behavior, and citizen science. Our objective in this article is to provide an approachable, end-to-end guide to help researchers apply image-based machine learning methods effectively to their own research problems. Using a case study, we describe how to prepare data, train and deploy models, and overcome common issues that can cause models to underperform. Importantly, we discuss how to diagnose problems that can cause poor model performance on new imagery to build robust tools that can vastly accelerate data acquisition in the marine realm. Code to perform analyses is provided at [https://github.com/](https://github.com/)[ ](https://github.com/)heinsense2/AIO_CaseStudy.

FAQs

What is the main purpose of this review for researchers?

To provide an approachable end-to-end guide that helps researchers apply image-based machine learning methods effectively to their own problems. It uses a case study to explain preparation, training, deployment, and troubleshooting.

Which types of tasks does image-based machine learning aim to automate?

It aims to classify content in images, localize and count objects, and partition images into meaningful regions. These mirror tasks humans perform through visual interpretation.

Why do machine learning models sometimes underperform on new imagery?

Common issues related to model performance on new images are discussed, including problems that can be diagnosed to build more robust tools. The review emphasizes overcoming factors that limit generalization.

Aug 2, 2026

Upload by Patrick