[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117991-en":3,"doc-seo-117991-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117991,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","MULTIIOT - Benchmarking Machine Learning for the Internet of Things","Next-generation machine learning systems must perceive and interact with the physical world through diverse sensory channels. In IoT, motion, thermal, geolocation, depth, wireless signals, video, and audio are used to model environments and human states, yet the community lacks broad benchmarks that support large-scale training across many sensors and tasks. MULTIIOT is proposed as a unified benchmark with 1.15M+ samples across 12 modalities and 8 real-world tasks, spanning multimodal generalization, long-range interactions, sensor heterogeneity, and training/inference complexity.","arXiv :2311 .062 17v2 [ cs .LG] 4 Jul 2024  \nMULTIIOT: Benchmarking Machine Learning  \nfor the Internet of Things  \nShentong Mo, Louis-Philippe Morency, Ruslan Salakhutdinov, Paul Pu Liang  \nCarnegie Mellon University  \n[shentongmo@gmail.com](shentongmo@gmail.com)  \nAbstract  \nThe next generation of machine learning systems must be adept at perceiving and interacting with the physical world through a diverse array of sensory channels.  \nCommonly referred to as the ‘Internet of Things (IoT)’ ecosystem, sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments and the humans inside them. Despite the potential for understanding human wellbeing, controlling physical devices, and interconnecting smart cities, the community has seen limited benchmarks for building machine learning systems for IoT. Existing efforts are often specialized to a single sensory modality or prediction task, which makes it difficult to study and train large-scale models across many IoT sensors and tasks.  \nTo accelerate the development of new machine learning technologies for IoT, this paper proposes MULTIIOT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks. MULTIIOT introduces unique challenges involving (1) generalizable learning from many sensory modalities,(2) multimodal interactions across long temporal ranges,(3) extreme heterogeneity due to unique structure and noise topologies in real-world sensors, and (4) complexity during training and inference. We evaluate a comprehensive set of models on MULTIIOT, including modality and task-specific methods, multisensory and multitask supervised models, and large multisensory foundation models. Our results highlight opportunities for ML to make a significant impact in IoT, but many challenges in scalable learning from heterogeneous, longrange, and imperfect sensory modalities still persist. We release all code and data at the repository 1 to accelerate future research in machine learning for IoT.  \n1 Introduction  \nThe next generation of machine learning systems will need to understand and interact with the physical world through physical sensors. This interconnection of sensors is typically called the Internet of Things (IoT) ecosystem, which includes motion, thermal, geolocation, depth, wireless signals, pose, video, and audio to model the states of physical environments and the humans inside them [7, 35, 52] . These sensing technologies have had great impact in recognizing human physical activities to inform us of our daily physical wellness [38, 49, 57]; navigating self-driving cars and efficiently connecting them with transportation grids [25, 28]; and recognizing if humans require assistance in schools, hospitals, or the workplace [1, 3, 31] .  \nWhile the field of machine learning for IoT has great potential, existing efforts are often specialized to a single sensory modality or prediction task [6, 27, 32, 11], resulting in limited resources to systematically study large-scale learning across many IoT sensors and tasks. To standardize the benchmarking and development of new machine learning technologies for IoT, this paper proposes MULTIIOT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples covering 12 real-world sensory modalities and 8 IoT tasks firmly rooted in practical scenarios  \n1[https://github.com/Multi-IoT/MultiIoT](https://github.com/Multi-IoT/MultiIoT)  \nPreprint. Under review.  \nFigure 1: MULTII OT is the largest benchmark for machine learning on the Internet of Things (IoT), consisting of 1.15M samples, 12 rich modalities, and 8 challenging tasks such as perceiving the pose, gaze, activities, and gestures of humans as well as the touch, contact, pose, and 3D structure of physical objects. MULTIIOT presents new challenges of (1) generalizable learning from many senso","cbCailMTe8igIEhP","https://ap.wps.com/l/cbCailMTe8igIEhP","pdf",2930359,1,30,"English","en",105,"# Introduction\n## MULTIIOT Overview and Benchmark Scope\n## Unique Challenges\n## Evaluation of Models\n## Release of Code and Data","[{\"question\":\"What does MULTIIOT include in terms of data scale and task coverage?\",\"answer\":\"MULTIIOT includes over 1.15 million samples, covering 12 sensory modalities and 8 real-world IoT tasks focused on practical scenarios such as human perception and physical object understanding.\"}]","MULTIIOT - Benchmarking Machine Learning for the Internet of Things | PDF",1785680667,76,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"multiiot-benchmarking-machine-learning-for-the-internet-of-things","",{"@graph":36,"@context":77},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/multiiot-benchmarking-machine-learning-for-the-internet-of-things/117991/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What does MULTIIOT include in terms of data scale and task coverage?","Question",{"text":75,"@type":76},"MULTIIOT includes over 1.15 million samples, covering 12 sensory modalities and 8 real-world IoT tasks focused on practical scenarios such as human perception and physical object understanding.","Answer","https://schema.org",{"og:url":52,"og:type":79,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":81,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":21,"slug":113},"research-report",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":98,"slug":129},19,"General","general"]