Building High-Quality Datasets for Information Retrieval Evaluation at a Reduced Cost

  1. David Otero 1
  2. Daniel Valcarce
  3. Javier Parapar 1
  4. Álvaro Barreiro 1
  1. 1 Universidade da Coruña
    info

    Universidade da Coruña

    La Coruña, España

    ROR https://ror.org/01qckj285

Libro:
XoveTIC 2019: The 2nd XoveTIC Conference (XoveTIC 2019), A Coruña, Spain, 5–6 September
  1. Alberto Alvarellos González (ed. lit.)
  2. José Joaquim de Moura Ramos (ed. lit.)
  3. Beatriz Botana Barreiro (ed. lit.)
  4. Javier Pereira Loureiro (ed. lit.)
  5. Manuel F. González Penedo (ed. lit.)

Editorial: MDPI

ISBN: 978-3-03921-444-0,  978-3-03921-443-3

Ano de publicación: 2019

Congreso: XoveTIC (2. 2019. A Coruña)

Tipo: Achega congreso

DIALNET GOOGLE SCHOLAR lock_openAcceso aberto editor

Resumo

Information Retrieval is not any more exclusively about document ranking. Continuously new tasks are proposed on this and sibling fields. With this proliferation of tasks, it becomes crucial to have a cheap way of constructing test collections to evaluate the new developments. Building test collections is time and resource consuming: it requires time to obtain the documents, to define theuser needs and it requires the assessors to judge a lot of documents. To reduce the latest, pooling strategies aim to decrease the assessment effort by presenting to the assessors a sample of documents in the corpus with the maximum number of relevant documents in it. In this paper, we propose the preliminary design of different techniques to easily and cheapily build high-quality test collections without the need of having participants systems.

Fonte de datos: Dialnet