Emotion Analysis using the interaction with Companion Robot in the depressed elderly in the community

Abstract

Purpose The companion Robots (CR) may have various and high usefulness for emotional support of the human. This is because robots can not only have simple social interactions with the humans, but also execute cognitive functions such as learning, situation inference, or planning through AI technology. To contribute to the development and improvement of CR for the older adults, we investigated the accuracy of emotion analysis for the elderly using conversation and touch reaction with CR called “Hyodol”, which has a soft material like a doll. Method This study was conducted on 200 depressed seniors in the community using CR from February 1 to April 30, 2022.  Participants were asked to express their emotions (positive or negative) by touching the CR twice a day and briefly talk about the reason within 15 seconds. The voice conversations were recorded automatically with consent. Data of participants who responded more than 45 times (25%) out of 180 conversations over 3 months were used for analysis. We classified emotions as ‘positive’, ‘negative’, and ‘neutral’ in consideration of the context by directly listening to the voice data. This human-classified emotional data conducted by researchers was compared with the emotions of the classification results by CR touch response and compared them with emotions obtained using the Naver AI-based sentiment analysis program (Table 1). Results and Discussion 1,763 data obtained from 22 participants were analyzed. Positive and negative emotions by the CR touch response showed 98.69% and 81.34%, respectively, consistent with human-classified emotions. The concordance rate of AI sentiment analysis with the human-classified emotions was 54.43% for positive responses and 62.04% for negative responses. For the neutral responses that could not be identified by CR Touch response, 325(18.43%) were found for human-classified and 469 (26.63%) for AI analysis. There were 32 data (24.24%) that AI mistakenly recognized ambient noise (TV sound, phone calls, etc.) as the response of the senior, and there were 76 data (4.66%) analyzed by AI that the elderly’s conversational voice was unresponsive. Classifying the senior's emotion using CR touch had a higher concordance rate with the actual emotional state than using Korea’s most recent AI. Since AI sentiment training data was mainly built based on the language habits of the young generation in their 20s and 30s, there are limitations to reflect the language habits of the senior who express emotion in various ways using irony, local dialects, and non-verbal expressions such as sighs and laughter. For the development of interaction and emotional support services of CR for the older adults, it is necessary to construct a dataset that can reflect the actual language and expression habits of the senior. In addition, it may be required to improve the recognition and processing of noise common around the elderly and to apply non-verbal communication approaches familiar to the elderly.

Keywords: Companion robot, older adult, AI, Voice data, Emotion analysis

Address: Department of Psychiatry, Yonsei University Wonju College of Medicine, Republic of Korea

Email: gooddr@yonsei.ac.kr

Acknowledgement This research was supported by the national Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2022M3E8A1081453), and the core technology development project for reorganization of new industries through the Korea Institute for Advancement of Technology (KIAT), funded by the Ministry of Trade, Industry and Energy (MOTIE) (grant number: P0018674).