Please use this identifier to cite or link to this item:
Publication type: Conference paper
Type of review: Peer review (publication)
Title: CEASR : a corpus for evaluating automatic speech recognition
Authors : Ulasik, Malgorzata Anna
Hürlimann, Manuela
Germann, Fabian
Gedik, Esin
Benites de Azevedo e Souza, Fernando
Cieliebak, Mark
et. al : No
DOI : 10.21256/zhaw-20125
Proceedings: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020)
Editors of the parent work: Calzolari, Nicoletta
Béchet, Frédéric
Blache, Philippe
Choukri, Khalid
Cieri, Christopher
Declerck, Thierry
Goggi, Sara
Isahara, Hitoshi
Maegaard, Bente
Mariani, Joseph
Mazo, Hélène
Moreno, Asuncion
Odijk, Jan
Piperidis, Stelios
Pages : 6477
Pages to: 6485
Conference details: 12th Language Resources and Evaluation Conference (LREC) 2020
Issue Date: 2020
Publisher / Ed. Institution : European Language Resources Association
ISBN: 979-10-95546-34-4
Language : English
Subjects : Automatic speech recognition; Evaluation; Speech corpus; ASR system
Subject (DDC) : 004: Computer science
Abstract: In this paper, we present CEASR, a Corpus for Evaluating ASR quality. It is a data set derived from public speech corpora, containing manual transcripts enriched with metadata along with transcripts generated by several modern state-of-the-art ASR systems. CEASR provides this data in a unified structure, consistent across all corpora and systems with normalised transcript texts and metadata. We then use CEASR to evaluate the quality of ASR systems on the basis of their Word Error Rate (WER). Our experiments show, among other results, a substantial difference in quality between commercial versus open-source ASR tools and differences up to a factor of ten for single systems on different corpora. By using CEASR, we could very efficiently and easily obtain these results. This shows that our corpus enables researchers to perform ASR-related evaluations and various in-depth analyses with noticeably reduced effort: without the need to collect, process and transcribe the speech data themselves.
Fulltext version : Published version
License (according to publishing contract) : CC BY-NC 4.0: Attribution - Non commercial 4.0 International
Departement: School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Appears in Collections:Publikationen School of Engineering

Files in This Item:
File Description SizeFormat 
2020_Ulasik-etal_CEASR_LREC.pdf733.4 kBAdobe PDFThumbnail

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.