Crawling the client-side hidden web

  1. Álvarez, Manuel 1
  2. Pan, Alberto 1
  3. Raposo, Juan 1
  4. Viña, Ángel 1
  1. 1 Universidade da Coruña
    info

    Universidade da Coruña

    La Coruña, España

    ROR https://ror.org/01qckj285

Libro:
Proceedings of the IADIS International Conference WWW/INTERNET 2004: Madrid, Spain, October 6-9, 2004
  1. Isaías, Pedro (coord.)
  2. Karmakar, Nitya (coord.)

Editorial: IADIS (International Association for Development of the Information Society)

ISBN: 972-99353-0-0

Año de publicación: 2004

Título del volumen: Short Papers-Posters

Volumen: 2

Páginas: 1179-1182

Congreso: International Conference on WWW/Internet (3. 2004. Madrid)

Tipo: Aportación congreso

Resumen

There is a great amount of information on the web that can not be accessed by conventional crawler engines. This portion of the web is usually called hidden web data. To be able to deal with this problem, it is necessary to solve two tasks: crawling the client-side and crawling the server-side hidden web. In this paper we present an architecture and a set of related techniques for assessing the information placed in the client-side hidden web, dealing with aspects such as JavaScript technology, non-standard session maintenance mechanisms, client redirections, pop-up menus, etc. Our approach leverages current browser APIs and implements novel crawling models and algorithms.