Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases

Fathi, Kiavash; Darvishy, Alireza; van de Venn, Hans Wernher

doi:10.1109/ICCE-Berlin56473.2022.9937109

Publication type:	Conference paper
Type of review:	Peer review (publication)
Title:	Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases
Authors:	Fathi, Kiavash Darvishy, Alireza van de Venn, Hans Wernher
et. al:	No
DOI:	10.1109/ICCE-Berlin56473.2022.9937109
Proceedings:	2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin)
Conference details:	12th IEEE International Conference on Consumer Technology (ICCE-Berlin), Berlin, Germany, 2-6 September 2022
Issue Date:	2022
Publisher / Ed. Institution:	IEEE
ISBN:	978-1-6654-5676-0
Language:	English
Subjects:	Assistive technology; Mobility; Augmented reality; Scene understanding; Spatial mapping; Path planning
Subject (DDC):	006: Special computer methods
Abstract:	With the Augmented Reality (AR) technology available today, it is quite feasible to accommodate the needs of the visually impaired (VI) via AR. In this paper, a framework is introduced to help the VI navigate and explore unfamiliar indoor environments. In contrast to commonly used AR applications focused on visual augmentation, the proposed framework employs auditory three-dimensional feedback (A3DF) for guiding the VI. Concretely, the current framework reads the pose of the user and helps the VI reach a target location via A3DF. The A3DF is implemented with the Unity game engine to provide the optimal user experience. After acquiring the environment mesh (EM), the optimal path from the user’s location to the target location is calculated, while avoiding obstacles using Unity’s navigation system. Moreover, the user is provided with semantic information about the unknown environment whilst exploring via auditory information. This framework is implemented on Microsoft HoloLens 2 and tested at an office environment with different locations of interest. Additionally, this framework potentially accelerates the learning curve since the user can be trained using Unity’s simulation environment. Lastly, given different design parameters of the framework, the proposed method can be fine-tuned to fit the specific needs of the individual
URI:	https://digitalcollection.zhaw.ch/handle/11475/26126
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering
Organisational Unit:	Institute of Computer Science (InIT) Institute of Mechatronic Systems (IMS)
Appears in collections:	Publikationen School of Engineering

Files in This Item:

There are no files associated with this item.

Show full item record

Fathi, K., Darvishy, A., & van de Venn, H. W. (2022). Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases. 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin). https://doi.org/10.1109/ICCE-Berlin56473.2022.9937109

Fathi, K., Darvishy, A. and van de Venn, H.W. (2022) ‘Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases’, in 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin). IEEE. Available at: https://doi.org/10.1109/ICCE-Berlin56473.2022.9937109.

K. Fathi, A. Darvishy, and H. W. van de Venn, “Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases,” in 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin), 2022. doi: 10.1109/ICCE-Berlin56473.2022.9937109.

FATHI, Kiavash, Alireza DARVISHY und Hans Wernher VAN DE VENN, 2022. Augmented reality for the visually impaired : navigation aid and scene semantics for indoor use cases. In: 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin). Conference paper. IEEE. 2022. ISBN 978-1-6654-5676-0

Fathi, Kiavash, Alireza Darvishy, and Hans Wernher van de Venn. 2022. “Augmented Reality for the Visually Impaired : Navigation Aid and Scene Semantics for Indoor Use Cases.” Conference paper. In 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin). IEEE. https://doi.org/10.1109/ICCE-Berlin56473.2022.9937109.

Fathi, Kiavash, et al. “Augmented Reality for the Visually Impaired : Navigation Aid and Scene Semantics for Indoor Use Cases.” 2022 IEEE 12th International Conference on Consumer Electronics (ICCE-Berlin), IEEE, 2022, https://doi.org/10.1109/ICCE-Berlin56473.2022.9937109.