Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-20318
Publication type: Article in scientific journal
Type of review: Editorial review
Title: Survey on evaluation methods for dialogue systems
Authors : Deriu, Jan Milan
Rodrigo, Alvaro
Otegi, Arantxa
Echegoyen, Guillermo
Rosset, Sophie
Agirre, Eneko
Cieliebak, Mark
et. al : No
DOI : 10.1007/s10462-020-09866-x
10.21256/zhaw-20318
Published in : Artificial Intelligence Review
Issue Date: 2020
Publisher / Ed. Institution : Springer
ISSN: 0269-2821
1573-7462
Language : English
Subjects : Dialogue systems; Artificial intelligence; Evaluation; Deep learning
Subject (DDC) : 004: Computer science
Abstract: In this paper, we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation, in and of itself, is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost- and time-intensive. Thus, much work has been put into finding methods which allow a reduction in involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented, conversational, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then present the evaluation methods regarding that class.
URI: https://digitalcollection.zhaw.ch/handle/11475/20318
Fulltext version : Published version
License (according to publishing contract) : CC BY 4.0: Attribution 4.0 International
Departement: School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Published as part of the ZHAW project : LIHLITH - Learning to Interact with Humans by Lifelong Interaction with Humans
Appears in Collections:Publikationen School of Engineering

Files in This Item:
File Description SizeFormat 
2020_Deriu-etal_Survey-on-evaluation-methods-for-dialogue-systems.pdf1.94 MBAdobe PDFThumbnail
View/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.