Towards a Testing Framework for Machine Learning Model Deployment in Manufacturing Systems

Aug 3, 2026

Upload by Miles

Machine learning model deployment in manufacturing systems brings challenges that require robust testing to guarantee reliable, efficient operation. The paper presents an automated testing framework that verifies correct use of data sources, validates model functionality, and evaluates compatibility between the deployed model and the target machine. A literature review establishes existing approaches and highlights gaps, while the framework organizes unit, integration, regression, and performance tests to match manufacturing-specific needs.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

Introduction

State of the Art

Running the model on the target machine

[Available online at www.sciencedirect.com](Available online at www.sciencedirect.com) ScienceDirect Procedia CIRP 127 (2024) 122–128 10th CIRP Conference on Assembly Technology and Systems (CIRP CATS 2024) Towards a Testing Framework for Machine Learning Model Deployment in Manufacturing Systems I. Heidera, *, J. Baumgärtnera, A. Botta, R. Ströbela, A. Puchtaa, J. Fleischera awbk Institute of Production Science, Kaiserstr. 12, 76131 Karlsruhe, Germany * Corresponding author. Tel.: +49 172 141 1977 ; fax: +0-000-000-0000. E-mail address: [imanuel.heider@kit.edu](imanuel.heider@kit.edu) Abstract The deployment of machine learning models in manufacturing systems presents unique challenges, necessitating robust testing procedures to ensure reliable and efficient operation. This paper proposes an automated testing framework specifically designed to address these challenges, focusing on verifying the correct utilization of data sources, validating model functionality, and assessing the compatibility of the target machine with the deployed model. By automating the testing process, this framework aims to enhance the reliability and effectiveness of machine learning model deployment in manufacturing systems. Through a comprehensive literature review, the paper explores existing methodologies and identifies gaps in current practices. The proposed framework incorporates various test types, including unit tests, integration tests, regression tests, and performance tests, each tailored to the specific requirements of manufacturing systems. Experimental results demonstrate the framework’s effectiveness in detecting errors and failures during the deployment process. Overall, this research contributes to advancing the field of machine learning deployment in manufacturing systems and provides practical insights for practitioners seeking to optimize the reliability and efficiency of their deployed models. © 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ([https://creativecommons.org/licenses/by-nc-nd/4.0](https://creativecommons.org/licenses/by-nc-nd/4.0)) Peer-review under responsibility of the scientific committee of the 18th CIRP Conference on Computer Aided Tolerancing Keywords: Type your keywords here, separated by semicolons ; 1. Introduction The digitization of the production landscape has led to the proliferation of data-driven solutions in manufacturing systems. New machines are equipped with ever more sensors collecting data at high frequencies, this increasing availability of data has enabled the development of advanced machine learning (ML) models. But especially in small and mediumsized companies, most machines do not have these so-called Industry 4.0 capabilities [4] . This has led to a retrofitting market, with a plethora of companies offering solutions for the integration of additional sensor technology and software tools for data connection to machines and their controls [13] . These new systems can also enable the deployment of machine- learning models. However, most of these systems are vendorspecific and are neither transferable nor scalable. Current development trends, therefore, try to automate the integration into a seamless process. However, in even in these cases, the main focus lies with established ML application areas (i.e. the domain of ‘Big Tech‘) where computing power and data are readily available. They do not consider the specific requirements of machine learning models in a manufacturing context. This paper aims to close this gap by proposing a testing framework specifically designed to allow for risk-free deployment of machine learning models into manufacturing systems. The framework is designed to be easily integrated into existing development processes and to be scalable to different machine-learning models. 2212-8271 © 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND

FAQs

What is the main goal of the proposed testing framework?

To enable risk-free deployment of machine learning models into manufacturing systems by systematically testing data usage, model behavior, and target-machine compatibility.

Which types of tests does the framework include?

It incorporates unit tests, integration tests, regression tests, and performance tests, each tailored to manufacturing system requirements.

Why is testing especially important for deploying to manufacturing edge or embedded devices?

Systems with limited computing power require assurance that the model runs stably on the target machine and can handle target-machine data, which may not be confirmed without dedicated testing.

Aug 3, 2026

Upload by Miles