Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features

Neururer, Daniel; Dellwo, Volker; Stadelmann, Thilo

doi:10.1016/j.patrec.2024.03.016

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-30407

Full metadata record

DC Field	Value	Language
dc.contributor.author	Neururer, Daniel	-
dc.contributor.author	Dellwo, Volker	-
dc.contributor.author	Stadelmann, Thilo	-
dc.date.accessioned	2024-03-27T15:49:48Z	-
dc.date.available	2024-03-27T15:49:48Z	-
dc.date.issued	2024-03-26	-
dc.identifier.issn	0167-8655	de_CH
dc.identifier.issn	1872-7344	de_CH
dc.identifier.uri	https://digitalcollection.zhaw.ch/handle/11475/30407	-
dc.description.abstract	While deep neural networks have shown impressive results in automatic speaker recognition and related tasks, it is dissatisfactory how little is understood about what exactly is responsible for these results. Part of the success has been attributed in prior work to their capability to model supra-segmental temporal information (SST), i.e., learn rhythmic-prosodic characteristics of speech in addition to spectral features. In this paper, we (i) present and apply a novel test to quantify to what extent the performance of state-of-the-art neural networks for speaker recognition can be explained by modeling SST; and (ii) present several means to force respective nets to focus more on SST and evaluate their merits. We find that a variety of CNN- and RNN-based neural network architectures for speaker recognition do not model SST to any sufficient degree, even when forced. The results provide a highly relevant basis for impactful future research into better exploitation of the full speech signal and give insights into the inner workings of such networks, enhancing explainability of deep learning for speech technologies.	de_CH
dc.language.iso	en	de_CH
dc.publisher	Elsevier	de_CH
dc.relation.ispartof	Pattern Recognition Letters	de_CH
dc.rights	https://creativecommons.org/licenses/by/4.0/	de_CH
dc.subject	Speaker verification	de_CH
dc.subject	Speaker clustering	de_CH
dc.subject	Dynamic feature	de_CH
dc.subject	Prosodic features	de_CH
dc.subject	Deep learning	de_CH
dc.subject	Explainable AI (XAI)	de_CH
dc.subject.ddc	006: Spezielle Computerverfahren	de_CH
dc.title	Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features	de_CH
dc.type	Beitrag in wissenschaftlicher Zeitschrift	de_CH
dcterms.type	Text	de_CH
zhaw.departement	School of Engineering	de_CH
zhaw.organisationalunit	Centre for Artificial Intelligence (CAI)	de_CH
dc.identifier.doi	10.1016/j.patrec.2024.03.016	de_CH
dc.identifier.doi	10.21256/zhaw-30407	-
zhaw.funding.eu	No	de_CH
zhaw.originated.zhaw	Yes	de_CH
zhaw.pages.end	69	de_CH
zhaw.pages.start	64	de_CH
zhaw.publication.status	publishedVersion	de_CH
zhaw.volume	181	de_CH
zhaw.publication.review	Peer review (Publikation)	de_CH
zhaw.webfeed	Datalab	de_CH
zhaw.webfeed	Machine Perception and Cognition	de_CH
zhaw.webfeed	ZHAW digital	de_CH
zhaw.author.additional	No	de_CH
zhaw.display.portrait	Yes	de_CH
Appears in collections:	Publikationen School of Engineering

Files in This Item:

File	Description	Size	Format
2024_Neururer_Deep-learning-networks-for-automatic-speaker-recognition_VoR.pdf	Published Version	1.28 MB	Adobe PDF	View/Open
2024_Neururer_Deep-learning-networks-for-automatic-speaker-recognition.pdf	Accepted Version	6.76 MB	Adobe PDF	View/Open

Show simple item record

Neururer, D., Dellwo, V., & Stadelmann, T. (2024). Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features. Pattern Recognition Letters, 181, 64–69. https://doi.org/10.1016/j.patrec.2024.03.016

Neururer, D., Dellwo, V. and Stadelmann, T. (2024) ‘Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features’, Pattern Recognition Letters, 181, pp. 64–69. Available at: https://doi.org/10.1016/j.patrec.2024.03.016.

D. Neururer, V. Dellwo, and T. Stadelmann, “Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features,” Pattern Recognition Letters, vol. 181, pp. 64–69, Mar. 2024, doi: 10.1016/j.patrec.2024.03.016.

NEURURER, Daniel, Volker DELLWO und Thilo STADELMANN, 2024. Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features. Pattern Recognition Letters. 26 März 2024. Bd. 181, S. 64–69. DOI 10.1016/j.patrec.2024.03.016

Neururer, Daniel, Volker Dellwo, and Thilo Stadelmann. 2024. “Deep Neural Networks for Automatic Speaker Recognition Do Not Learn Supra-Segmental Temporal Features.” Pattern Recognition Letters 181 (March): 64–69. https://doi.org/10.1016/j.patrec.2024.03.016.

Neururer, Daniel, et al. “Deep Neural Networks for Automatic Speaker Recognition Do Not Learn Supra-Segmental Temporal Features.” Pattern Recognition Letters, vol. 181, Mar. 2024, pp. 64–69, https://doi.org/10.1016/j.patrec.2024.03.016.