Please use this identifier to cite or link to this item:
https://doi.org/10.21256/zhaw-18985| Publication type: | Working paper – expertise – study |
| Title: | Survey on evaluation methods for dialogue |
| et. al: | No |
| Editors: | Deriu, Jan Milan Rodrigo, Alvaro Otegi, Arantxa Guillermo, Echegoyen Rosset, Sophie Agirre, Eneko Cieliebak, Mark |
| DOI: | 10.21256/zhaw-18985 |
| Extent: | 62 |
| Issue Date: | 10-May-2019 |
| Publisher / Ed. Institution: | ZHAW Zürcher Hochschule für Angewandte Wissenschaften |
| Other identifiers: | arXiv: 1905.04071v1 |
| Language: | English |
| Subject (DDC): | 006: Special computer methods |
| Abstract: | In this paper we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost and time intensive. Thus, much work has been put into finding methods, which allow to reduce the involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented dialogue systems, conversational dialogue systems, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then by presenting the evaluation methods regarding this class. |
| URI: | https://arxiv.org/abs/1905.04071v1 https://digitalcollection.zhaw.ch/handle/11475/18985 |
| License (according to publishing contract): | CC BY 4.0: Attribution 4.0 International |
| Departement: | School of Engineering |
| Organisational Unit: | Institute of Computer Science (InIT) |
| Appears in collections: | Publikationen School of Engineering |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| arxiv_1905.04071.pdf | Survey on Evaluation Methods for Dialogue Systems | 776.65 kB | Adobe PDF |  View/Open |
Show full item record
Deriu, J. M., Rodrigo, A., Otegi, A., Guillermo, E., Rosset, S., Agirre, E., & Cieliebak, M. (2019). Survey on evaluation methods for dialogue. ZHAW Zürcher Hochschule für Angewandte Wissenschaften. https://doi.org/10.21256/zhaw-18985
Deriu, J.M. et al. (eds) (2019) Survey on evaluation methods for dialogue. ZHAW Zürcher Hochschule für Angewandte Wissenschaften. Available at: https://doi.org/10.21256/zhaw-18985.
J. M. Deriu et al., Eds., “Survey on evaluation methods for dialogue,” ZHAW Zürcher Hochschule für Angewandte Wissenschaften, May 2019. doi: 10.21256/zhaw-18985.
DERIU, Jan Milan, Alvaro RODRIGO, Arantxa OTEGI, Echegoyen GUILLERMO, Sophie ROSSET, Eneko AGIRRE und Mark CIELIEBAK (Hrsg.), 2019. Survey on evaluation methods for dialogue, 2019. [online]. ZHAW Zürcher Hochschule für Angewandte Wissenschaften. Verfügbar unter: https://arxiv.org/abs/1905.04071v1
Deriu, Jan Milan, Alvaro Rodrigo, Arantxa Otegi, Echegoyen Guillermo, Sophie Rosset, Eneko Agirre, and Mark Cieliebak, eds. 2019. “Survey on Evaluation Methods for Dialogue.” ZHAW Zürcher Hochschule für Angewandte Wissenschaften. https://doi.org/10.21256/zhaw-18985.
Deriu, Jan Milan, et al., editors. Survey on Evaluation Methods for Dialogue. ZHAW Zürcher Hochschule für Angewandte Wissenschaften, 10 May 2019, https://doi.org/10.21256/zhaw-18985.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.