Title: Black box evaluation for operational information retrieval applications
Authors : Braschler, Martin
Imhof, Melanie
Rietberger, Stefan
Published in : Bridging Between Information Retrieval and Databases
Pages : 192
Pages to: 207
Publisher / Ed. Institution : Springer
Publisher / Ed. Institution: Heidelberg
Issue Date: 2013
License (according to publishing contract) : Licence according to publishing contract
Series : Lecture Notes in Computer Science
Series volume: 8173
Type of review: Not specified
Language : English
Subjects : IE; INIT
Subject (DDC) : 020: Library and information sciences
Abstract: The black box application evaluation methodology described in this tutorial is applicable to a broad range of operational information retrieval (IR) applications. Contrary to popular, traditional IR evaluation approaches that are limited to measure the IR system performance on a test collection, the black box evaluation methodology considers an IR application in its entirety: the underlying system, the corresponding document collection, and its configuration/application layer. A comprehensive set of quality criteria is used to estimate the user’s perception of the application. Scores are assigned as a weighted average of results from tests that evaluate individual aspects. The methodology was validated in a small evaluation campaign. An analysis of this campaign shows a correlation between the testers’ perception of the applications and the evaluation scores. Moreover, functional weaknesses of the tested IR applications can be identified and then systematically targeted.
Departement: School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Publication type: Book Part
DOI : 10.1007/978-3-642-54798-0_9
ISBN: 978-3-642-54797-3
URI: https://digitalcollection.zhaw.ch/handle/11475/4499
Appears in Collections:Publikationen School of Engineering

Files in This Item:
There are no files associated with this item.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.