[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85317-en":3,"doc-seo-85317-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85317,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Benchmarking Edge Inference Strategies for Deep Learning Models in Industrial Machine Vision","Edge deployment is a common choice for industrial machine vision when low latency, data security, and limited connectivity are essential. While multiple frameworks exist to optimize inference on edge devices, few studies benchmark their inference-time behavior under realistic industrial deployment conditions. This comparative study evaluates plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT across CPU and GPU platforms using both CNNs and a transformer-based vision model.","Benchmarking Edge Inference Strategies for Deep Learning Models in Industrial Machine Vision  \nMiguel Gomez Fernandez  \nSmart Systems and Smart Manufacturing (S3M) AIMENO Porri˜no, Spain [https://orcid.org/0009-0008-0433-9710](https://orcid.org/0009-0008-0433-9710)  \nRoi Mendez-Rial  \nSmart Systems and Smart Manufacturing (S3M) AIMENO Porri˜no, Spain [https://orcid.org/0000-0002-9991-0316](https://orcid.org/0000-0002-9991-0316)  \nDavid Castro Boga  \nSmart Systems and Smart Manufacturing (S3M) AIMENO Porri˜no, Spain [https://orcid.org/0000-0002-1329-770X](https://orcid.org/0000-0002-1329-770X)  \nEric Lopez-Lopez  \nSmart Systems and Smart Manufacturing (S3M) AIMENO Porri˜no, Spain [https://orcid.org/0000-0002-7720-1607](https://orcid.org/0000-0002-7720-1607)  \narXiv :2607 . 11356v1 [ cs .CV] 13 Jul 2026  \nAbstract—Edge deployment is often the preferred solution for industrial machine vision systems when low latency, data security, or limited connectivity are critical requirements. Several frameworks are available to optimise inference on edge devices; however, relatively few studies have systematically compared their inference-time performance under industrial deployment conditions.  \nIn this work, we present a comparative study of four widely used approaches for machine vision inference in industrial settings: plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT. The evaluation focuses on inference time, covers several CPU- and GPU-based hardware platforms, and includes both conventional convolutional neural networks and a transformerbased vision model. For the hardware platforms and models evaluated, the results show that OpenVINO achieves the lowest inference time on CPUs, while TensorRT achieves the lowest inference time on GPUs. However, TensorRT does not always outperform plain PyTorch for the transformer-based model considered in this study.  \nIndex Terms—Edge computing, Industrial machine vision, Deep neural networks, Inference optimisation, Performance benchmarking, Embedded AI  \nI. INTRODUCTION  \nDeep learning has strong practical value in industrial machine vision, particularly for tasks such as quality control, precise alignment, and security monitoring [1], [2] . These scenarios often impose strict requirements on latency, data control, and connectivity, making cloud-based inference less  \nThe research leading to these results has received funding from “Proyecto Desarrollo de una Estrategia integral de REciCladO de BATer´ıas - RECOBATS”, as part of the TransMisiones 2024 initiative, under project file number PLEC2024-011135 . This project is funded by the Spanish Ministry of Science, Innovation and Universities (MICIU), the Spanish State Research Agency (AEI), and the European Regional Development Fund (FEDER, EU), under reference MICIU/AEI/10.13039/501100011033/FEDER, UE, in cooperation with CDTI. This work has also received funding from the Horizon Europe programme under grant agreement No. 101298421 (GRAIL) .  \nViews and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA) . Neither the European Union nor the granting authority can be held responsible for them.  \nappropriate. For this reason, edge deployment is frequently the preferred solution in industrial machine vision systems [3]–[6] .  \nHowever, despite these advantages, edge deployments must operate under tighter constraints in terms of computing power, memory, and energy consumption. As a result, considerable effort is devoted to optimising the inference of a given model for the target hardware. Besides model-level techniques such as quantisation, pruning, or knowledge distillation [7], [8], inference performance can also be improved through deployment frameworks that provide hardware-specific optimisations and more efficient execution backends. Selecting the most suitable deployment framework is therefore a practical challenge, si","cbCaijCgyfFvCRny","https://ap.wps.com/l/cbCaijCgyfFvCRny","pdf",2542917,1,6,"English","en",105,"# Introduction\n## Motivation and challenges of edge inference\n## Deployment frameworks and the need for benchmarking\n## Prior work and identified research gap","[{\"question\":\"Why is edge deployment preferred in industrial machine vision?\",\"answer\":\"Edge deployment is preferred when systems require low latency, data security, and limited connectivity for industrial machine vision tasks.\"},{\"question\":\"Which four inference frameworks are compared in this study?\",\"answer\":\"The study compares plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT for edge inference in industrial settings.\"},{\"question\":\"How do the results differ across CPU and GPU platforms?\",\"answer\":\"OpenVINO achieves the lowest inference time on CPUs for the evaluated setups, while TensorRT achieves the lowest inference time on GPUs. For the transformer-based model, TensorRT does not always outperform plain PyTorch.\"}]",1784202446,15,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"benchmarking-edge-inference-strategies-for-deep-learning-models-in-industrial-machine-vision","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/benchmarking-edge-inference-strategies-for-deep-learning-models-in-industrial-machine-vision/85317/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is edge deployment preferred in industrial machine vision?","Question",{"text":75,"@type":76},"Edge deployment is preferred when systems require low latency, data security, and limited connectivity for industrial machine vision tasks.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which four inference frameworks are compared in this study?",{"text":80,"@type":76},"The study compares plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT for edge inference in industrial settings.",{"name":82,"@type":73,"acceptedAnswer":83},"How do the results differ across CPU and GPU platforms?",{"text":84,"@type":76},"OpenVINO achieves the lowest inference time on CPUs for the evaluated setups, while TensorRT achieves the lowest inference time on GPUs. For the transformer-based model, TensorRT does not always outperform plain PyTorch.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]