Infrared thermography has shown great promise as a diagnostic method for health care, providing useful information on a person's physiological, and pathological state. Recently, the use of artificial intelligence combined with infrared technology has boosted the adoption of thermal imaging in various applications and has been proposed to recognize human emotion by measuring facial skin temperature. However, its application has been limited to laboratory settings due to demanding computational and hardware resources. In this scenario, this work presents the design and development of a portable system based on a low power microcontroller implementing an optimized Edge-AI solution for binary emotional state classification using minimal hardware resources. The recognition of happiness and sadness emotional states induced by audiovisual stimuli serves as a case-study for feasibility assessment. Thermal images, produced by an uncooled and low-cost thermal sensor, along with electrocardiogram, are acquired and processed with an Arm® Cortex®-M4 microcontroller. A simple, yet effective neural network has been developed, optimized, and deployed to run the emotion detection algorithm in real time. The complete system has been experimentally verified and results in terms of accuracy and hardware constraints are discussed. Specifically, by employing a dataset consisting of 60 infrared videos, an accuracy of 80% was achieved with a resource occupation of 3.4 kB of RAM and 76.4 kB of flash memory.
A Microcontroller-Based System for Human-Emotion Recognition with Edge-AI and Infrared Thermography / Gragnaniello, M.; Borghese, A.; Marrazzo, V. R.; Breglio, G.; Irace, A.; Riccio, M.. - 1110:(2024), pp. 327-332. (Intervento presentato al convegno International Conference on Applications in Electronics Pervading Industry, Environment and Society, APPLEPIES 2023 tenutosi a ita nel 2023) [10.1007/978-3-031-48121-5_46].
A Microcontroller-Based System for Human-Emotion Recognition with Edge-AI and Infrared Thermography
Gragnaniello M.;Borghese A.;Marrazzo V. R.;Breglio G.;Irace A.;Riccio M.
2024
Abstract
Infrared thermography has shown great promise as a diagnostic method for health care, providing useful information on a person's physiological, and pathological state. Recently, the use of artificial intelligence combined with infrared technology has boosted the adoption of thermal imaging in various applications and has been proposed to recognize human emotion by measuring facial skin temperature. However, its application has been limited to laboratory settings due to demanding computational and hardware resources. In this scenario, this work presents the design and development of a portable system based on a low power microcontroller implementing an optimized Edge-AI solution for binary emotional state classification using minimal hardware resources. The recognition of happiness and sadness emotional states induced by audiovisual stimuli serves as a case-study for feasibility assessment. Thermal images, produced by an uncooled and low-cost thermal sensor, along with electrocardiogram, are acquired and processed with an Arm® Cortex®-M4 microcontroller. A simple, yet effective neural network has been developed, optimized, and deployed to run the emotion detection algorithm in real time. The complete system has been experimentally verified and results in terms of accuracy and hardware constraints are discussed. Specifically, by employing a dataset consisting of 60 infrared videos, an accuracy of 80% was achieved with a resource occupation of 3.4 kB of RAM and 76.4 kB of flash memory.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
