[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120678-en":3,"doc-seo-120678-105":30,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120678,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253",6,"Technology","Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM","Edge deployment of AI faces a mismatch between training-oriented ML frameworks and the constrained execution environment of bare-metal devices. This paper presents an end-to-end code generator that parses a pre-trained model and emits C source libraries for MicroTVM, enabling ahead-of-time runtime execution with reduced latency and dependency. The work analyzes operator placement, showing compute-intensive operators can be offloaded via a UMA accelerator interface while remaining operators run on CPU cores. A hand-gesture recognition experiment on ARM Cortex-M4F validates the approach.","arXiv :2304 .04842v2 [ cs .LG] 14 Apr 2023  \nDeploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM  \nChen Liu􀀃 , Matthias Jobst􀀃y , Liyuan Guo􀀃 , Xinyue Shi􀀃 , Johannes Partzsch􀀃y , Christian Mayr􀀃y  \n􀀃 Chair of Highly-Parallel VLSI-Systems and Neuro-Microelectronics Technische Universita¨t Dresden  \nDresden, Germany  \nEmail: chen.liu@tu-dresden.de  \ny Centre for Tactile Internet with Human-in-the-loop (CeTI)  \nTechnische Universita¨t Dresden  \nDresden, Germany  \nAbstract—In the past few years, more and more AI applications have been applied to edge devices. However, models trained by data scientists with machine learning frameworks, such as PyTorch or TensorFlow, can not be seamlessly executed on edge. In this paper, we develop an end-to-end code generator parsing a pre-trained model to C source libraries for the backend using MicroTVM, a machine learning compiler framework extension addressing inference on bare metal devices. An analysis shows that specific compute-intensive operators can be easily offloaded to the dedicated accelerator with a Universal Modular Accelerator (UMA) interface, while others are processed in the CPU cores. By using the automatically generated ahead-of-time C runtime, we conduct a hand gesture recognition experiment on an ARM Cortex M4F core.  \nIndex Terms—TVM, MicroTVM, model deployment, BYOC, UMA  \nI. INTRODUCTION  \nMachine learning has been significantly advanced in the past decade. With a rising number of AI solutions in consumer and industrial products, AI tends to move closer to the edge, where is nearby the source of raw data. Different from the typical training environment in the data center, edge devices focus more on inference, but under rigorous restrictions in terms of processing time, memory and power consumption, which hamper model deployment.  \nTo bring AI to the edge, massive human efforts, such as graph optimization and quantization are mainly required. This process is prone to cause errors, to omit unconsidered operators, and is hard to be replicated on other devices, as it relies on the varied APIs provided by different backend chip vendors. Particularly when dedicated accelerators are involved  \nThis work is funded by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany's Excellence Strategy – EXC 2050/1 – Project ID 390696704 – Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop”(CeTI) of Technische Universitt Dresden.  \nThis paper was accepted at the CODAI'22 workshop, co-hosted with Embedded Systems Week, after it underwent a double-blind peer review process.  \nto speed up specific operations, which is observed more and more common, model deployment becomes more complicated.  \nMachine learning compiler frameworks alleviate this issue. They don't attempt to adapt the architecture of the model, but bridge the gap between a pre-trained model and a backend device by parsing, optimizing, and compiling the model graph, together with parameters, to executable. Some sizable enterprises are keen on developing their own compiler frameworks, such as TensorFlow Lite (TFLite) [1], PyTorch Glow [2], TVM [3] and OpenVINO [4] . These frameworks all support multiple input model formats and parse the model graph to high-level intermediate representatives (IR), then lower them to low-level IRs, which target heterogeneous backends like CPU, GPU, FPGA, and machine learning ASICs. However, a dynamical interpretation of the model graph is regarded as an inefficient way, especially on bare-metal Internet of things (IoT) devices. Ahead-of-time (AOT) compilation, asthe alternative, draws booming interest due to less latency and more concise dependency. TensorFlow Lite Micro (TFLM) [5] is the framework addressing AOT compilation on embedded systems among the TensorFlow toolbox, but requires C++11 support and comprises limited operators. Accordingly, TVM provides MicroTVM for the same sake.  \nIn this paper we propose a Micr","cbCaimSsOr2AJjxu","https://ap.wps.com/l/cbCaimSsOr2AJjxu","pdf",893682,1,4,"English","en",105,"# Introduction\n## Edge deployment constraints and motivation\n## Compiler frameworks and AOT compilation\n# Implementation\n## End-to-end C code generator design\n## Frontends and model format support\n## Structured C output and build integration\n# Method overview\n## Unsupported operator registration strategy\n## UMA-based operator offloading\n# Evaluation\n## Real-time hand gesture recognition experiment","[{\"question\":\"What problem does the MicroTVM-based approach address for edge devices?\",\"answer\":\"It bridges the gap between pre-trained models built in common ML frameworks and bare-metal edge backends where execution must meet strict time, memory, and power constraints.\"},{\"question\":\"How does the paper handle operators that are not supported by the target backend?\",\"answer\":\"It introduces an operator registration strategy so the input model can be parsed completely and compiled for the backend runtime.\"},{\"question\":\"What role does UMA play in the deployment workflow?\",\"answer\":\"UMA provides a interface to offload specific compute-intensive operator patterns to the dedicated on-chip accelerator, while other parts execute on CPU cores.\"}]","Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM | PDF",1785731363,10,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":28},"deploying-machine-learning-models-to-ahead-of-time-runtime-on-edge-using-microtvm","",{"@graph":36,"@context":84},[37,53,67],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/technology/",3,{"item":52,"name":13,"@type":43,"position":21},"https://docshare.wps.com/document/deploying-machine-learning-models-to-ahead-of-time-runtime-on-edge-using-microtvm/120678/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the MicroTVM-based approach address for edge devices?","Question",{"text":74,"@type":75},"It bridges the gap between pre-trained models built in common ML frameworks and bare-metal edge backends where execution must meet strict time, memory, and power constraints.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the paper handle operators that are not supported by the target backend?",{"text":79,"@type":75},"It introduces an operator registration strategy so the input model can be parsed completely and compiled for the backend runtime.",{"name":81,"@type":72,"acceptedAnswer":82},"What role does UMA play in the deployment workflow?",{"text":83,"@type":75},"UMA provides a interface to offload specific compute-intensive operator patterns to the dedicated on-chip accelerator, while other parts execute on CPU cores.","https://schema.org",{"og:url":52,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,112,117,122,127,130,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":46,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":110,"slug":111},50,"technology",{"id":113,"doc_module":4,"doc_module_name":46,"category_name":114,"show_sort_weight":115,"slug":116},7,"Healthcare",40,"healthcare",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":119,"show_sort_weight":120,"slug":121},8,"Research & Report",30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":29,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":29,"slug":132},"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]