[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-126004-en":3,"doc-seo-126004-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},126004,687207024643,"Oliver","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","A Machine Learning based Real-Time Application for Engagement Detection - Research Report","Human engagement estimation has expanded rapidly with the growth of smart computing devices and the need for interfaces that can perceive attentional states. The work proposes a real-time application using a single RGB camera to infer engagement by combining facial expression recognition and gaze-direction analysis. Fast models are built with dlib and custom residual neural networks, trained on a modified DAiSEE dataset of multi-label affective states from 112 real-world subjects. Experiments evaluate robustness and effectiveness despite the lack of an external baseline.","A Machine Learning based Real-Time Application for Engagement Detection  \nEmanuele Iacobelli 1 , Samuele Russo2 and Christian Napoli1,3,4  \n1 Department of Computer, Control and Management Engineering, Sapienza University of Rome, 00185 Roma, Italy;  \n2 Department of Psychology, Sapienza University of Rome, 00185 Roma, Italy;  \n3 Institute for Systems Analysis and Computer Science, Italian National Research Council, 00185 Roma, Italy;  \n4 Department of Computational Intelligence, Czestochowa University of Technology, 42 -201 Czestochowa, Poland;  \nAbstract  \nThe study of human engagement has significantly grown in recent years, particularly accelerated by the interaction with a growing number of smart computing machines [1, 2, 3] . Engagement estimation has significant importance across various domains of study, including advertising, marketing, human-computer interaction, and healthcare [4, 5, 6] . In this paper, we propose a real-time application that leverages a single RGB camera to capture user behavior. Our approach implementsa novel method for estimating human engagement in real-world scenarios by extracting valuable information from the combination of facial expressions and gaze direction analysis. To acquire this data, we employed fast and accurate machine learning algorithms from the external library dlib, along with custom versions of Residual Neural Networks implemented from scratch. For training our models, we used a modified version of the DAiSEE dataset, a multi-label user affective states classification dataset that collects frontal videos of 112 different people recorded in real-world scenarios. In the absence of a baseline for comparing the results obtained by our application, we conducted experiments to assess its robustness in estimating engagement levels, leading to very encouraging results.  \nKeywords  \nEngagement Detection, Eye Tracking, Face Expression Recognition, Machine Learning, Residual Neural Networks  \n1. Introduction  \nIn today’s rapidly evolving digital landscape, humanity interacts with a growing number of smart computing machines. This situation highlights the increasing trend of direct interactions with smart devices in various domains, including household assistance, customer service, and industrial applications. Despite this technological advancement, many devices lack algorithms capable of perceiving and responding to users’ attentional states. Traditional user interfaces still heavily rely on explicit input or predefined triggers, resulting in often inefficient and mechanical interactions.  \nThe potential for automatic acquisition and interpretation of users’ engagement represents a huge usability improvement for Human-Computer Interaction (HCI) and Human-Robot Interaction (HRI) systems. This capability holds the promise of ushering in more advanced and intuitive interactions, elevating system responsiveness, and enhancing overall user experience. In detail, engagement is a fundamental aspect of the human experience and captures in depth the quality of an individual’s involve-  \nSYSYEM 2023: 9th Scholar’s Yearly Symposium of Technology, Engi neering and Mathematics, Rome, December 3-6, 2023  \n[Envelope-Open](Envelope-Open iacobelli@diag.uniroma1.it)[ iacobelli@diag.uniroma1.it](Envelope-Open iacobelli@diag.uniroma1.it) (E. Iacobelli); [samuele.russo@uniroma1.it](samuele.russo@uniroma1.it) (S. Russo); [cnapoli@diag.uniroma1.it](cnapoli@diag.uniroma1.it)[ ](cnapoli@diag.uniroma1.it)(C. Napoli)  \nOrcid 0009-0003-1379-9106 (E. Iacobelli); 0000-0002-1846-9996 (S. Russo); 0000-0002-3336-5853 (C. Napoli)  \n© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4 .0 International (CC BY 4 .0) .  \n75  \nCEUR ~~  ~~[Workshop](Workshop ceur-ws.org)[ ceur-ws.org](Workshop ceur-ws.org)[ ](Workshop ceur-ws.org)[Proceedings](Proceedings ISSN 1613-0073)[ ISSN 1613-0073](Proceedings ISSN 1613-0073)   \nment, focus, and interaction with their surroundings. Fo","cbCaifh1rxQIBhAm","https://ap.wps.com/l/cbCaifh1rxQIBhAm","pdf",1693483,3,1,10,"English","en",105,"# Abstract\n# Keywords\n# 1. Introduction\n## Motivation for engagement-aware HCI/HRI\n## Proposed real-time pipeline\n## Dataset construction and preprocessing\n## Experimental robustness evaluation","[{\"question\":\"What input does the proposed engagement detection application use?\",\"answer\":\"It relies on a single RGB camera to capture user behavior in real-world interaction scenarios.\"},{\"question\":\"How does the method estimate engagement level?\",\"answer\":\"It combines facial expression analysis with gaze direction estimation, then merges both pipeline outputs using a weighted linear interpolation.\"},{\"question\":\"How were the machine learning models trained and evaluated?\",\"answer\":\"Models were trained on a modified DAiSEE multi-label affective-states dataset, with preprocessing to reduce video computational costs. Robustness and effectiveness are assessed through experiments despite the absence of a baseline for direct comparison.\"}]","A Machine Learning based Real-Time Application for Engagement Detection - Research Report | PDF",1785902510,25,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"a-machine-learning-based-real-time-application-for-engagement-detection-research-report","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/a-machine-learning-based-real-time-application-for-engagement-detection-research-report/126004/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-16","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What input does the proposed engagement detection application use?","Question",{"text":76,"@type":77},"It relies on a single RGB camera to capture user behavior in real-world interaction scenarios.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does the method estimate engagement level?",{"text":81,"@type":77},"It combines facial expression analysis with gaze direction estimation, then merges both pipeline outputs using a weighted linear interpolation.",{"name":83,"@type":74,"acceptedAnswer":84},"How were the machine learning models trained and evaluated?",{"text":85,"@type":77},"Models were trained on a modified DAiSEE multi-label affective-states dataset, with preprocessing to reduce video computational costs. Robustness and effectiveness are assessed through experiments despite the absence of a baseline for direct comparison.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":22,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":22,"slug":134},"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]