Scalability and Efficiency Challenges in Large-Scale Web Search Engines

  • Scalability and Efficiency Challenges in Large-Scale Web Search Engines
  • Monday, 7th, 9:00~17:30
  • Room 300
  • Presenters: B. Barla Cambazoglu (Yahoo! Research), Ricardo Baeza-Yates (Yahoo! Research)
  • Abstract
    The main goals of a web search engine are quality, efficiency, and scalability. In this tutorial, we focus on the last two goals, providing a fairly comprehensive overview of the scalability and efficiency challenges in large-scale web search engines. In particular, the tutorial provides an in-depth architectural overview of a web search engine, mainly focusing on the web crawling, indexing, and query processing components. The scalability and efficiency issues encountered in these components are presented at four different granularities: at the level of a single computer, a cluster of computers, a single data center, and a multi-center search engine. The tutorial also points at open research problems and provides recommendations to researchers who are new to the field.
Menu