Sofía González-Villalobos, Yanwei Miao, Julia Renardias, Mehrbod Manavi (2023): Erinnern Sie sich noch? A Storytelling Game with Pepper for Older Adults’ Memory Training. 9th International Conference on Infrastructures in Healthcare. DOI: 10.48340/ihc2023_pd026 Erinnern Sie sich noch? A Storytelling Game with Pepper for Older Adults’ Memory Training Sofía González-Villalobos, Yanwei Miao, Julia Renardias, Mehrbod Manavi University of Siegen sofia.gvillalobos@student.uni-siegen.de, yanwei.miao@student.uni-siegen.de, julia.renardias@student.uni-siegen.de mehrbod.manavi@uni-siegen.de Abstract. Providing care for older adults in Germany is becoming increasingly challenging due to current demographic and social trends. To address this issue, technologies, such as robot-based, are being used to develop and test new ideas on how to provide nursing care. Moreover, many older adults can be affected by memory loss problems. Memory training can play a crucial role in helping ameliorate these issues, improve memory performance, and connect with others. This project aims to offer a memory game that provides residents with a meaningful activity, such as memory training, that they can enjoy and in parallel it may support nursing staff and caregivers in a retirement home through robot-based assistance. Introduction Germany’s current demographic and social trends pose increasing challenges to providing care for older adults (Statisches Bundesamt, 2022). In response to this, digital technologies are being harnessed to explore innovative approaches in nursing care provision. The objective is not to replace face-to-face care work but rather to enhance and supplement it (Abdi et al., 2018). Given the complex nature of this situation, the role of academia has become indispensable in analyzing potential strategies to support the health of older adults, including the utilization of robot-based assistance (Carros et al., 2020). This technology aims to provide social companionship to older adults, in addition to the physical aid typically provided by caregivers (Broekens et al., 2009). Furthermore, it holds the potential to improve the well-being of older adults and alleviate the workload of caregiving staff (Kachouie, 2014). Moreover, memory training has been recognized as a crucial element in assisting older adults in mitigating memory loss problems (APA, 2022), enhancing their memory performance (Blue et al., 2020), and fostering social connections (Broekens et al., 2009). This student project is situated within the broader context of the e-ViTA (EU-Japan Virtual Coach for Smart Ageing) (e-ViTA, 2023) research initiative. e-ViTA research aims to develop virtual coaching devices that empower fit older adults to be more active and healthy (Naccarelli et al., 2023). The storytelling memory training game “Erinnern Sie sich noch?” primary objective is to extend support to nursing personnel and caregivers within a retirement home through the implementation of robot-assisted aid. In addition, the project seeks to enhance the quality of life for residents by offering engaging and purposeful activities, such as memory training. These activities are designed not only for enjoyment but also to foster motivation and potential collaboration among residents. Related work The study explored key aspects related to older adults’ memories, medical causes, professional practices, and the potential of technology for memory training, including robot-based technology for support. Aging can lead to changes in specific brain areas, affecting personal reflections on the past and future during conversations (Wank et al., 2020). Memory difficulties in seniors may stem from the accumulation of a vast volume of memories rather than an inability to remember (Thompson et al., 2022). Recall and sharing of memories are important for older adults, promoting brain training, social connection, and meaning in life. Reminiscing is considered beneficial in psychological practice with older adults, as highlighted in the American Psychological Association’s guidelines (APA, 2022). Reminiscence therapy, utilizing life histories, has shown positive effects. The development of technology, such as smartphone-based cognitive assessment, offers precise estimation and tracking of cognitive abilities, potentially aiding in early detection of conditions like Alzheimer’s disease (Grilli, University of Arizona, 2020). Robot-based assistance is another strategy to support older adults’ health, providing social companionship and reducing caregivers’ workload (Carros et al., 2020; Broekens et al., 2009; Kachouie et al., 2014). These insights and the potential of technology have influenced our design concept. 2 Methodology Data collection and analysis To deepen our comprehension, we conducted a sequence of semi-structured interviews involving 4 older adults and 1 caregiver residing in a care home. The primary objectives were to elicit insights into the following dimensions: 1) The daily routines and experiences of older adults living in a care facility, 2) The catalysts that evoke and sustain their memories, shedding light on the mechanisms of retention, 3) The individuals with whom seniors opt to share their reminiscences, along with the underlying considerations, and 4) The level of trust older adults place in technology, particularly their perceptions of Pepper, the robotic assistant. Importantly, most residents and caregivers in this facility are familiar with Pepper due to prior University of Siegen studies (Carros et al., 2020; Schwaninger et al., 2023; Carros et al., 2022). This background sets the stage for our exploration of their perceptions and experiences in this research. All participants received an explanation of the study aims and signed informed consent documents about their participation. The participant demographics were recorded to provide a better understanding of the population under study (Table I). Participant s Interview ID Age Gender Role Familiarity with Pepper P1 1 ~65 F Older adult familiar P2 1 88 M Older adult familiar P3 2 ~65 F Older adult familiar P4 3 92 F Older adult not familiar P5 4 N/A F Caregiver familiar Table I. Participants of semi-structure interviews Findings Upon transcribing the interviews, we used a thematic analysis approach, which led to the identification of themes across diverse dimensions of the interview questions. Regarding daily routines and experiences of older adults living in a care facility, it revealed that most older adults displayed a keen interest towards engaging in physical activities, irrespective of their physical limitations. Furthermore, a notable proportion of participants enjoyed socializing over afternoon coffee with a small group of friends, providing an opportunity for engaging in discussions. Turning to memories and reminiscing, it emerged that all older adults frequently engaged in reminiscence, displaying a strong desire to share their 3 cherished life moments with others. Among the various topics discussed, beautiful travel memories emerged as the most popular. Notably, photographs served as the primary method for archiving and preserving these treasured memories. Lastly, with regard to the perception of technology trustworthiness and attitudes towards Pepper, certain older adults perceived robots as mere machines, associating their function solely with manual tasks. Nevertheless, a significant majority found Pepper’s humanoid appearance to be appealing and expressed a genuine affinity towards it. Concept & testing Based on the insights garnered, it becomes apparent that there exists a desire to share memories among friends within the care facility. Simultaneously, this presents an opportune moment to create an interactive and fun memory training game suitable for their leisure time. Consequently, several requisites were identified, the conceptual framework ought to be tailored for small group contexts, as interactions with peers have the potential to stimulate active participation among older adults. Moreover, it should actively promote the recall of specific memories that engage participants emotionally, thus estimulating cognitive capacities. Additionally, the program must take into account popular interests and inclinations of older individuals, such as trips, to curate a personalized memory training experience. Lastly, a focus should be placed on building a trusting and reliable rapport between participants and the Pepper robot. Subsequently, a game concept was formulated, capitalizing on the interactive capabilities of the Pepper robot to attain its goals. This game entails Pepper showcasing pictures of shared experiences of the residents (e.g., care home-organized summer trips) on its tablet. Moreover, Pepper narrates these experiences, prompts participants with specific questions to recall details from their memories employs voice recognition technology to distinguish correct and incorrect responses, and provides continuous encouragement. Furthermore, the humanoid appearance of Pepper, complemented with crafted dialogues, along with positive tones and body language, seeks to establish a sense of friendship and familiarity, thereby motivating older adults to perceive Pepper as a companion. The pictures employed in the games were from group activities held by the nursing home, where all participants have taken part, no private pictures, nor sensitive data included. It is important to note that the nursing home willingly provided both the pictures and the associated details for the service, maintaining a commitment to ethical practices and data sharing. 4 The prototype of the concept featured two distinct storylines based on past experiences, such as a trip to a zoo1 and a small celebration2 of the Cologne carnival in the nursing home. The game utilized images from these activities, and implemented by the Choreograph software and installed on the robot. Choregraphe is a visual programming language implemented by Softbank robotics to program the NAO and Pepper robot (Subedi et al., 2021; Pot et al., 2009). An interface for the tablet was designed using HTML and CSS to showcase the event pictures and support storytelling. In order to enhance the concept, a series of testing sessions were carried out. The majority of these sessions took place in a robotics laboratory to address initial concerns including adjusting voice recognition sensitivity and iterating on the interface and storyline design. Following this, the game was introduced to a different nursing home for additional testing sessions with residents using Pepper. These sessions were scheduled during leisure hours, gathering further feedback and valuable insights. It is noteworthy to mention that the residents and caregivers at this facility possess minimal to no familiarity with Pepper. Discussion The interviews with residents and caregivers provided valuable insights shaping the program’s concept and implementation. However, the sample size was limited and it considers the perspective of a single care facility. Participants’ attitudes during testing sessions indicated notable curiosity and engagement with Pepper, as one participant stated, “Now that’s something different!” Nevertheless, as mentioned before a broader participant pool is necessary for a more comprehensive evaluation. The humanoid appearance of Pepper was well-received, fostering a friendly atmosphere. Additionally, the tailored storylines using pictures and brief notes received a positive response, prompting lively discussions about past experiences. Moreover, the older adults seemed motivated to receive positive reinforcement for answering quiz questions correctly. However, it was discussed that newcomers or individuals without past group activities may face challenges engaging with the game. This due to their lack of familiarity and/or absence of personal pictures. Furthermore, displaying pictures of recently deceased individuals may cause confusion and emotional distress, requiring further research on how participants perceive such situations, especially residents with dementia or diverse emotional vulnerabilities. 2 The nursing home hosted a traditional Cologne’s carnival celebration where the residents dressed up, gathered together, and enjoyed the festival while also learning about the carnival’s history. 1 The nursing home organized a trip to the zoo, where residents had the opportunity to visit, socialize, and learn interesting facts about the animals there. 5 Limitations surfaced in the context of direct interactions with the robot, often requiring assistance for comprehending instructions and maintaining smooth conversations. Although the "Help" option was programmed to assist residents by providing hints without providing a solution, managing interactions within group settings remained intricate. This complexity occasionally led to frustration among participants, as expressed by one individual’s remark, “...(robots) are something for the future...”. Adjustments to the voice recognition system are necessary to accommodate older adults’ voice capabilities. Pepper’s functionalities could be constrained when addressing issues like visual impairment or difficulties in adapting to technology. The role of caregivers support and the potential for multimodal interactions may serve to address such challenges. Ethical considerations are vital in implementing digital technologies for older adults, prioritizing their needs, autonomy, and right to refuse technology use. Hence, while Pepper could serve as a companion, thereby helping to alleviate the limited caregiver availability in nursing homes, its purpose is to support the care provided by them rather than replace human interaction or the role of caregivers. Conclusion The Pepper robot-incorporated storytelling memory game exhibits potential in enhancing memory and cognitive function among older adults through visual reminiscing. However, a comprehensive, extensive observation involving a larger group of participants is necessary to determine its impact on brain performance. Memory games and social activities hold great importance for seniors in retirement homes. However, staff shortages may result in activity cancellations. Encouragingly, participants warmly welcomed Pepper as a potential companion, thus offering a possible opportunity to support caregivers. Consequently, the integration of robotics and consistent cognitive training could potentially alleviate this challenge. Looking ahead, transitioning from sole voice recognition to a multi-modal approach, could provide an opportunity to better suit diverse older adult capabilities, augmenting the concept’s care home support. References Abdi J., Al-Hindawi A., Ng T., and Vizcaychipi M. P. (2018): ‘Scoping review on the use of socially assistive robot technology in elderly care’. BMJ Open, 8(2). American Psychological Association. (2022): ‘Guidelines for Psychological Practice with Older Adults’, Retrieved May 26, 2023 from https://www.apa.org/practice/guidelines/older-adults 6 Aubrey A. Wank, Matthias R. Mehl, Jessica R. Andrews-Hanna, Angelina J. Polsinelli, Suzanne Moseley, Elizabeth L. Glisky, and Matthew D. Grilli. (2020): ‘Eavesdropping on Autobiographical Memory: A Naturalistic Observation Study of Older Adults’ Memory Sharing in Daily Conversations.’ Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2020.00238 Broekens J., Heerink M., and Rosendal H. (2009): ‘Assistive social robots in elderly care: A review’, Gerontechnology, 8:94–103, 04. Carros F., Meurer J., Lüffler D., Unbehaun D., Matthies S., Koch I., Wieching R., Randall D., Hassenzahl M., and Wulf V. (2020): ‘Exploring human-robot interaction with the elderly: Results from a ten-week case study in a care home’ In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI ’20, pp. 1–12, New York, NY, USA. Association for Computing Machinery. Carros F., Schwaninger I., Preussner A., Randall D., Wieching R., Fitzpatrick G., and Wulf V. (2022): ‘Care Workers Making Use of Robots: Results of a Three-Month Study on Human-Robot Interaction within a Care Home’, In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 631, pp. 1–15. https://doi.org/10.1145/3491102.3517435 Demografische Aspekte. (2022): Retrieved May 3, 2023 from https://www.destatis.de/DE/Themen/Querschnitt/Demografischer-Wandel/Aspekte/demografi e-bevoelkerungszahl.html. e-ViTA. (2023): ‘EU-Japan Virtual coach for smart ageing’, Retrieved May 1, 2023 from https://www.e-vita.coach Kachouie R., Sedighadeli S., Khosla R., and Chu Mei-Tai. (2014): ‘Socially assistive robots in elderly care: A mixed-method systematic literature review.’ International Journal of Human–Computer Interaction, 30(5):369–393. Naccarelli R., Casaccia S., Homma K., Bevilacqua R. and Revel G. M. (2023): ‘e-VITA Use Cases Configurator: A Tool to Identify the Optimal Configuration of the Sensor Network and Coaching Devices to Enable Older People to Age Well at Home’ 2023 IEEE International Workshop on Metrology for Living Environment (MetroLivEnv), Milano, Italy, pp. 196-201, doi: 10.1109/MetroLivEnv56897.2023.10164067 Pot, E., Monceaux, J., Gelin, R., Maisonnier, B. (2009): ‘Choregraphe: a graphical tool for humanoid robot programming’ RO-MAN 2009, The 18th IEEE International Symposium on Robot and Human Interactive Communication, Toyama, Japan, pp. 46-51. https://doi.org/10.1109/ROMAN.2009.5326209 Schwaninger, I., Carros, F., Weiss, A. et al. (2023): ‘Video connecting families and social robots: from ideas to practices putting technology to work’ Univ Access Inf Soc 22, pp. 931–943. https://doi.org/10.1007/s10209-022-00901-y 7 Subedi, A., Pandey, D., Mishra, D. (2021): ‘Programming Nao as an Educational Agent: A Comparison Between Choregraphe and Python SDK’, In: Ben Ahmed, M., Boudhir, A.A., Karaș, İ.R., Jain, V., Mellouli, S. (eds): Innovations in Smart Cities Applications vol. 5, SCA, Lecture Notes in Networks and Systems. vol. 393, Springer, Cham. https://doi.org/10.1007/978-3-030-94191-8_29 Thompson D. (2022): ‘Clutter in the Attic: Why Memory Falters With Age’, Retrieved May 26, 2023 from https://consumer.healthday.com/2-14-clutter-in-the-attic-why-memory-falters-with-age-2656 546122.html University of Arizona. (2020): ‘Older adults share fewer memories as they age’, Retrieved May 26, 2023 from www.sciencedaily.com/releases/2020/06/200629140059.htm 8