| Publication type: | Book part |
| Type of review: | Editorial review |
| Title: | Black box evaluation for operational information retrieval applications |
| Authors: | Braschler, Martin Imhof, Melanie Rietberger, Stefan |
| DOI: | 10.1007/978-3-642-54798-0_9 |
| Published in: | Bridging between information retrieval and databases |
| Page(s): | 192 |
| Pages to: | 207 |
| Issue Date: | 2013 |
| Series: | Lecture Notes in Computer Science |
| Series volume: | 8173 |
| Publisher / Ed. Institution: | Springer |
| Publisher / Ed. Institution: | Heidelberg |
| ISBN: | 978-3-642-54797-3 |
| Language: | English |
| Subjects: | IE; INIT |
| Subject (DDC): | 020: Library and information sciences |
| Abstract: | The black box application evaluation methodology described in this tutorial is applicable to a broad range of operational information retrieval (IR) applications. Contrary to popular, traditional IR evaluation approaches that are limited to measure the IR system performance on a test collection, the black box evaluation methodology considers an IR application in its entirety: the underlying system, the corresponding document collection, and its configuration/application layer. A comprehensive set of quality criteria is used to estimate the user’s perception of the application. Scores are assigned as a weighted average of results from tests that evaluate individual aspects. The methodology was validated in a small evaluation campaign. An analysis of this campaign shows a correlation between the testers’ perception of the applications and the evaluation scores. Moreover, functional weaknesses of the tested IR applications can be identified and then systematically targeted. |
| URI: | https://digitalcollection.zhaw.ch/handle/11475/4499 |
| Fulltext version: | Published version |
| License (according to publishing contract): | Licence according to publishing contract |
| Departement: | School of Engineering |
| Organisational Unit: | Institute of Computer Science (InIT) |
| Appears in collections: | Publikationen School of Engineering |
Files in This Item:
There are no files associated with this item.
Show full item record
Braschler, M., Imhof, M., & Rietberger, S. (2013). Black box evaluation for operational information retrieval applications. In Bridging between information retrieval and databases (pp. 192–207). Springer. https://doi.org/10.1007/978-3-642-54798-0_9
Braschler, M., Imhof, M. and Rietberger, S. (2013) ‘Black box evaluation for operational information retrieval applications’, in Bridging between information retrieval and databases. Heidelberg: Springer, pp. 192–207. Available at: https://doi.org/10.1007/978-3-642-54798-0_9.
M. Braschler, M. Imhof, and S. Rietberger, “Black box evaluation for operational information retrieval applications,” in Bridging between information retrieval and databases, Heidelberg: Springer, 2013, pp. 192–207. doi: 10.1007/978-3-642-54798-0_9.
BRASCHLER, Martin, Melanie IMHOF und Stefan RIETBERGER, 2013. Black box evaluation for operational information retrieval applications. In: Bridging between information retrieval and databases. Heidelberg: Springer. S. 192–207. ISBN 978-3-642-54797-3
Braschler, Martin, Melanie Imhof, and Stefan Rietberger. 2013. “Black Box Evaluation for Operational Information Retrieval Applications.” In Bridging between Information Retrieval and Databases, 192–207. Heidelberg: Springer. https://doi.org/10.1007/978-3-642-54798-0_9.
Braschler, Martin, et al. “Black Box Evaluation for Operational Information Retrieval Applications.” Bridging between Information Retrieval and Databases, Springer, 2013, pp. 192–207, https://doi.org/10.1007/978-3-642-54798-0_9.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.