[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124941-en":3,"doc-seo-124941-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124941,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Multimodal emotion classification using machine learning in immersive and non-immersive virtual reality","Affective computing supports detection and recognition of emotional states, and this study uses machine learning to automatically classify emotions elicited in immersive versus non-immersive virtual reality. Emotional states were induced using film clips in VR, while participants’ physiological signals were recorded and analyzed to train classification models. Subjective ratings via emotional scales were collected for each clip to support parallel emotion recognition. Results show no significant immersion-dependent differences, and user-dependent models outperform user-independent ones for both data sources.","Virtual Reality (2024) 28:107  \n[https://doi.org/10.1007/s10055-024-00989-y](https://doi.org/10.1007/s10055-024-00989-y)  \nMultimodal emotion classification using machine learning in immersive and non‑immersive virtual reality  \nRodrigo Lima1,2,3 · Alice Chirico4 · RuiVarandas6,7 · Hugo Gamboa6,7 · Andrea Gaggioli4,5 ·  \nSergi Bermúdez i Badia1,2,3  \nReceived: 30 June 2022 / Accepted: 20 March 2024 / Published online: 6 May 2024 © The Author(s) 2024  \nAbstract  \nAffective computing has been widely used to detect and recognize emotional states. The main goal ofthis study was to detect emotional states using machine learning algorithms automatically. The experimental procedure involved eliciting emotional states using film clips in an immersive and non-immersive virtual reality setup. The participants’ physiological signals were recorded and analyzed to train machine learning models to recognize users’ emotional states. Furthermore, two subjective ratings emotional scales were provided to rate each emotional film clip. Results showed no significant differences between presenting the stimuli in the two degrees of immersion. Regarding emotion classification, it emerged that for both physiological signals and subjective ratings, user-dependent models have a better performance when compared to user-independent models. We obtained an average accuracy of 69.29 ± 11.41% and 71.00 ± 7.95% for the subjective ratings and physiological signals, respectively. On the other hand, using user-independent models, the accuracy we obtained was 54.0 ± 17.2% and 24.9 ± 4.0%, respectively. We interpreted these data as the result of high inter-subject variability among participants, suggesting the need for user-dependent classification models. In future works, we intend to develop new classification algorithms and transfer them to real-time implementation. This will make it possible to adapt to a virtual reality environment in real-time, according to the user’s emotional state.  \nKeywords Affective computing · Emotions · Wearables · Physiological signals · Machine learning · Virtual reality  \nAlice Chirico and Rui Varandas have contributed equally to this work.  \n* Rodrigo Lima [rodrigo.lima@arditi.pt](rodrigo.lima@arditi.pt)  \nAlice Chirico  \n[alice.chirico@unicatt.it](alice.chirico@unicatt.it)  \nRui Varandas  \n[r.varandas@campus.fct.unl.pt](r.varandas@campus.fct.unl.pt)  \nHugo Gamboa  \n[hgamboa@fct.unl.pt](hgamboa@fct.unl.pt)  \nAndrea Gaggioli  \n[andrea.gaggioli@unicatt.it](andrea.gaggioli@unicatt.it)  \nSergi Bermúdez i Badia  \n[sergi.bermudez@uma.pt](sergi.bermudez@uma.pt)  \n1 Faculdade de Ciências Exatas e Engenharia, Universidade da Madeira, Campus Universitário da Penteada, 9020-105 Funchal, Portugal  \n2 NOVA Laboratory for Computer Science and Informatics, Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, 2829-516 Caparica, Setúbal, Portugal  \n3 ARDITI-Agência Regional para o Desenvolvimento da Investigação, Tecnologia e Inovação, Caminho da Penteada, 9020-105 Funchal, Portugal  \n4 Dipartimento di Psicologia, Università Cattolica del Sacro Cuore, Largo Agostino Gemelli 1, 20123 Milan, Italy  \n5 Applied Technology for Neuro-Psychology Lab, I.R.C.C.S, Istituto Auxologico Italiano, Milan 20149, Italy  \n6 LIBPhys (Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics), Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, 2829-516 Caparica, Setúbal, Portugal  \n7 PLUX Wireless Biosignals S.A., Avenida 5 de Outubro 70, 1050-059 Lisboa, Portugal  \n1 Introduction  \nAffective computing (AC) is the computing that relates or influences emotions and focuses on understanding the psychophysiological mechanisms underlying the way humans recognize emotions (Bota et al. 2019) . Traditional methodologies to recognize emotions in AC usually rely on facial expressions and speech. However, these methods lack feasibility during in-field experiments (Chanel et al. 2007) .  \nPhysiological signals are an alternative meth","cbCaiuqBwJpiLNUM","https://ap.wps.com/l/cbCaiuqBwJpiLNUM","pdf",2770326,1,23,"English","en",105,"# Introduction\n## Problem background and motivation\n## Limitations of traditional approaches\n## Role of physiological signals and wearables\n## Virtual reality in emotion recognition\n# Research aims and questions\n# Experimental procedure","[{\"question\":\"How were emotional states elicited for model training and evaluation?\",\"answer\":\"Participants watched emotional film clips presented in immersive and non-immersive virtual reality setups.\"},{\"question\":\"What data sources were used for emotion classification?\",\"answer\":\"The study used physiological signals from participants and subjective ratings from emotional scales.\"},{\"question\":\"Which classification approach performed better: user-dependent or user-independent models?\",\"answer\":\"User-dependent models achieved better performance than user-independent models for both physiological signals and subjective ratings.\"}]","Multimodal emotion classification using machine learning in immersive and non-immersive virtual reality | PDF",1785895506,58,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"multimodal-emotion-classification-using-machine-learning-in-immersive-and-non-immersive-virtual-reality","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/multimodal-emotion-classification-using-machine-learning-in-immersive-and-non-immersive-virtual-reality/124941/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How were emotional states elicited for model training and evaluation?","Question",{"text":75,"@type":76},"Participants watched emotional film clips presented in immersive and non-immersive virtual reality setups.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What data sources were used for emotion classification?",{"text":80,"@type":76},"The study used physiological signals from participants and subjective ratings from emotional scales.",{"name":82,"@type":73,"acceptedAnswer":83},"Which classification approach performed better: user-dependent or user-independent models?",{"text":84,"@type":76},"User-dependent models achieved better performance than user-independent models for both physiological signals and subjective ratings.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]