Toward contactless human thermal monitoring - A framework for machine learning-based human thermo-physiology modeling augmented with computer vision

Aug 5, 2026

Upload by Finn

The transition to a human-centered indoor climate depends on knowing individuals’ thermal states at the body-part level, yet scalable personalization is hindered by intrusive wearable sensing and by privacy constraints. This work proposes non-intrusive personalized thermal sensing using multimodal IR and RGB inputs, integrating a Machine Learning-based Human Thermo-Physiology Model (ML-HTPM). Computer vision extracts thermal-comfort-relevant features (activity, clothing insulation, posture, age, sex) from RGB sequences, while IR uses body-part detection to estimate limited skin temperatures. The ML-HTPM is trained from JOS3-generated data with an LSTM prediction model, achieving RMSE below 0.5°C for most local skin temperatures.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Introduction

Indoor environmental quality and thermal comfort variability

Body temperature, skin temperature, and thermal sensation

Motivation for non-intrusive personalized sensing

Proposed non-intrusive framework

Multimodal IR and RGB sensing design

Machine Learning-based Human Thermo-Physiology Model (ML-HTPM)

Computer vision feature extraction from RGB

IR-based body-part detection and skin temperature estimation

Data generation and model training

JOS3-based data creation

LSTM prediction modeling

Results and performance

Local skin temperature accuracy (RMSE)

Conclusion

Building and Environment 245 (2023) 110850 | Toward contactless human thermal monitoring: A framework for Machine Learning-based human thermo-physiology modeling augmented with computer vision | | | | | --- | --- | --- | --- | | Mohamad Rida a, Mohamed Abdelfattahb, Alexandre Alahi b, Dolaana Khovalyga,∗ a Laboratory of Integrated Comfort Engineering (ICE), École polytechnique fédérale de Lausanne (EPFL), Fribourg, Switzerland
b Visual Intelligence for Transportation (VITA), École polytechnique fédérale de Lausanne (EPFL), Lausanne, Switzerland | | | | | A R T I C L E I N F O | | A B S T R A C T | | | Keywords:
Thermal comfort
Human thermo-physiology modeling Non-intrusive sensing
Artificial intelligence
Deep learning
Computer vision | | The transition towards a human-centered indoor climate is beneficial from occupants’ thermal comfort and from an energy reduction perspective. However, achieving this goal requires the knowledge of the thermal state of individuals at the level of body parts. Many current solutions rely on intrusive wearable technologies, which require physical access to individuals facing limitations in scalability. Personalizing the indoor environment demands increased sensing at individual levels presenting challenges in terms of data collection and ensuring privacy protection. To address this challenge, this paper introduces a novel approach to non-intrusive personalized humans thermal sensing that can acquire personal data while minimizing the amount of sensing required. The method investigates multi-modal sensing solutions based on IR and RGB images, and it includes the development of a Machine Learning-based Human Thermo-Physiology Model (ML-HTPM). With the help of computer vision, features important for thermal comfort such as activity level, clothing insulation, posture, age, and sex can be extracted from an RGB image sequence using models such as the SlowFast network, YOLOv 7, while limited skin temperatures can be extracted from an IR image using OpenPifPaf for body parts detection. The developed ML-HTPM is based on data generated from an open-source JOS3 model after applying a prediction model based on Long Short-Term Memory (LSTM). The results showed that a human thermo-physiology model using machine learning can be trained, showing an RMSE of less than 0.5 ◦ C in most of the local skin temperatures. | | 1. Introduction Rapid urbanization and the fact that people spend almost 90% of their time indoors makes indoor environmental quality (IEQ) accountable in new and existing buildings for assuring the well-being of the occupants [1]. IEQ is characterized by environmental categories such as thermal, air quality, lighting, and acoustics. While each category is important for the comfort and well-being of occupants, thermal (dis)comfort is the most familiar and easily recognizable by occupants. Therefore, the indoor temperatures are typically set in buildings with the aim of providing thermal comfort to occupants by keeping their sensation around thermal neutrality (the state when a human body primarily maintains its core body temperature with minimal metabolic regulation) [2,3]. However, the practice of setting the indoor temperature at a narrow range has resulted in almost 40% of operational energy use in buildings [4,5]. Surprisingly, as evidenced from multiple field studies, e.g., [6,7], the buildings that are supposedly designed ∗ Corresponding author. E-mail address: [dolaana.khovalyg@epfl.ch](dolaana.khovalyg@epfl.ch) (D. Khovalyg). according to the standardized requirements do not necessarily provide a satisfactory experience to their occupants primarily because of human diversity. The current methods of setting the indoor climate consider an average person [8], but ‘‘one size does not fit all’’ [9] and individual differences in thermal sensation are well documented in the literature [10,11]. Therefore, in quest of improving the well-being of occupants and l

FAQs

Why is contactless human thermal monitoring needed in indoor climate control?

Current personalization requires body-part thermal knowledge, but intrusive wearable technologies are hard to scale and raise privacy concerns. Contactless sensing enables individualized thermal-state estimation without physical access.

How does the proposed framework use RGB and IR data together?

RGB sequences processed by computer vision extract features tied to thermal comfort such as activity level, clothing insulation, posture, age, and sex. IR images provide limited skin temperature information for detected body parts using body-part detection.

What is ML-HTPM and how is it trained and evaluated?

ML-HTPM is a Machine Learning-based Human Thermo-Physiology Model that learns from JOS3 open-source simulation data. After applying an LSTM-based prediction model, evaluation shows RMSE under 0.5°C for most local skin temperatures.

Aug 5, 2026

Upload by Finn