Tutorial T3-M - Web Search Engine Metrics: Direct Metrics to Measure User Satisfaction
Ali Dasdan, Yahoo!
Kostas Tsioutsiouliklis, Yahoo!
Emre Velipasaoglu, Yahoo!
Abstract
Web search is an indispensible resource for users to search for information on the Web. Web search is also an important service for publishers and advertisers to present their content to users. Thus, user satisfaction is the key and must be quantified. In this tutorial, our goal is to give a practical review of web search metrics from a user satisfaction point of view. This viewpoint is intuitive enough for a typical user to express his or her web search experience using it. The metrics that we are planning to cover fall into the following areas: relevance, coverage, comprehensiveness, diversity, discovery freshness, content freshness, and presentation. We will also describe how these metrics can be mapped to proxy metrics for the stages of a generic search engine pipeline. Our hope is that practitioners can apply these metrics readily and that researchers can get problems to work on, especially in formalizing and refining metrics.
Presenter
Ali Dasdan is a director managing the metrics and analysis group of Yahoo! Web Search. His group is responsible for defining and implementing white-box metrics for production web search systems, production web search data, and the Web. His group is also responsible for monitoring the metrics and diagnosing the causes for anomalies detected. His research interests are in web search and advertising since 2006. He obtained his PhD in Computer Science from the University of Illinois at Urbana-Champaign in 1999.
Kostas Tsioutsiouliklis is a principal technical lead for the content relevance group of Yahoo! Web Search. His group is responsible for defining metrics for the production content systems, focusing mainly in the crawler and web graph systems, as well as designing policies to improve crawler and index tiering performance. His research interests are in web search in general since 2001. He obtained his PhD in Computer Science from Princeton University in 2002.
