Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction

Iwane, Fumiaki; Porssut, Thibault; Blanke, Olaf; Chavarriaga, Ricardo; Millan, Jose Del R.; Herbelin, Bruno; Boulic, Ronan

doi:10.1088/1741-2552/ad2c02

Publication type:	Article in scientific journal
Type of review:	Peer review (publication)
Title:	Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction
Authors:	Iwane, Fumiaki Porssut, Thibault Blanke, Olaf Chavarriaga, Ricardo Millan, Jose Del R. Herbelin, Bruno Boulic, Ronan
et. al:	No
DOI:	10.1088/1741-2552/ad2c02
Published in:	Journal of Neural Engineering
Issue Date:	22-Feb-2024
Publisher / Ed. Institution:	IOP Publishing
ISSN:	1741-2552 1741-2560
Language:	English
Subjects:	Brain computer interface; Break-in-embodiment; EEG; Error related potential; Reinforcement learning; Virtual reality
Subject (DDC):	006: Special computer methods
Abstract:	Objective. A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. Break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption. Approach. We propose and empirically demonstrate a novel Brain Computer Interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG data of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion. Main results. Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling "Plug-and-Play" ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports. Significance. We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.
URI:	https://digitalcollection.zhaw.ch/handle/11475/30277
Fulltext version:	Published version
License (according to publishing contract):	CC BY 4.0: Attribution 4.0 International
Departement:	School of Engineering
Organisational Unit:	Centre for Artificial Intelligence (CAI)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Iwane, F., Porssut, T., Blanke, O., Chavarriaga, R., Millan, J. D. R., Herbelin, B., & Boulic, R. (2024). Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction. Journal of Neural Engineering. https://doi.org/10.1088/1741-2552/ad2c02

Iwane, F. et al. (2024) ‘Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction’, Journal of Neural Engineering [Preprint]. Available at: https://doi.org/10.1088/1741-2552/ad2c02.

F. Iwane et al., “Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction,” Journal of Neural Engineering, Feb. 2024, doi: 10.1088/1741-2552/ad2c02.

IWANE, Fumiaki, Thibault PORSSUT, Olaf BLANKE, Ricardo CHAVARRIAGA, Jose Del R. MILLAN, Bruno HERBELIN und Ronan BOULIC, 2024. Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction. Journal of Neural Engineering. 22 Februar 2024. DOI 10.1088/1741-2552/ad2c02

Iwane, Fumiaki, Thibault Porssut, Olaf Blanke, Ricardo Chavarriaga, Jose Del R. Millan, Bruno Herbelin, and Ronan Boulic. 2024. “Customizing the Human-Avatar Mapping Based on EEG Error Related Potentials during Avatar-Based Interaction.” Journal of Neural Engineering, February. https://doi.org/10.1088/1741-2552/ad2c02.

Iwane, Fumiaki, et al. “Customizing the Human-Avatar Mapping Based on EEG Error Related Potentials during Avatar-Based Interaction.” Journal of Neural Engineering, Feb. 2024, https://doi.org/10.1088/1741-2552/ad2c02.