Tutorial T3-M - Web Search Engine Metrics: Direct Metrics to Measure User Satisfaction


Ali Dasdan, Yahoo!

Kostas Tsioutsiouliklis, Yahoo!

Emre Velipasaoglu, Yahoo!


Abstract

Web search is an indispensible resource for users to search for information on the Web. Web search is also an important service for publishers and advertisers to present their content to users. Thus, user satisfaction is the key and must be quantified. In this tutorial, our goal is to give a practical review of web search metrics from a user satisfaction point of view. This viewpoint is intuitive enough for a typical user to express his or her web search experience using it. The metrics that we are planning to cover fall into the following areas: relevance, coverage, comprehensiveness, diversity, discovery freshness, content freshness, and presentation. We will also describe how these metrics can be mapped to proxy metrics for the stages of a generic search engine pipeline. Our hope is that practitioners can apply these metrics readily and that researchers can get problems to work on, especially in formalizing and refining metrics.

Presenter

Ali Dasdan is a director managing the metrics and analysis group of Yahoo! Web Search. His group is responsible for defining and implementing white-box metrics for production web search systems, production web search data, and the Web. His group is also responsible for monitoring the metrics and diagnosing the causes for anomalies detected. His research interests are in web search and advertising since 2006. He obtained his PhD in Computer Science from the University of Illinois at Urbana-Champaign in 1999.

Kostas Tsioutsiouliklis is a principal technical lead for the content relevance group of Yahoo! Web Search. His group is responsible for defining metrics for the production content systems, focusing mainly in the crawler and web graph systems, as well as designing policies to improve crawler and index tiering performance. His research interests are in web search in general since 2001. He obtained his PhD in Computer Science from Princeton University in 2002.

Emre Velipasaoglu is a senior technical lead for the search relevance group of Yahoo! Web Search. His group is responsible for designing and developing the entire suite of ranking systems for the production web search. His research interests are in machine learning since 1991 and web search in general since 2005. He obtained his PhD in Electrical and Computer Engineering from Purdue University in 1997.


aetic ANEI autel si aeit Asociación Multisectorial de Empresas Españolas de Electrónica y Telecomunicaciones
Asociación de Técnicos de Informática bcs etsit EUITT Ministerio de Educación y Ciencia
ctic mid+ Ministerio de Educación y Ciencia Ministerio de Industria, Turismo y Comercio Ministerio de Administraciones Públicas

aetic ANEI autel si aeit