|
Tutorial title: Web Search Engine Metrics: Direct Metrics to Measure User Satisfaction
Venue: the 18th & 19th International World Wide Web Conferences (WWW 2009 & WWW 2010)
Abstract --
Web search is an indispensable resource for users to search for information on the Web. Web search is also an important service for publishers and advertisers to present their content to users. Thus, user satisfaction is the key and must be quantified. In this tutorial, our goal is to give a practical review of web search metrics from a user satisfaction point of view. This viewpoint is intuitive enough for a typical user to express his or her web search experience using it. The metrics that we are planning to cover fall into the following areas: relevance, coverage, diversity, discovery coverage and latency, content freshness, and presentation. We will also describe how these metrics can be mapped to proxy metrics for the stages of a generic search engine pipeline. Our hope is that practitioners can apply these metrics readily and that researchers can get problems to work on, especially in formalizing and refining metrics.
Presenters (2009 bios)
Ali Dasdan is a director managing the metrics and analysis group of Yahoo! Web Search. His group is responsible for defining and implementing white-box metrics for production web search systems, production web search data, and the Web. His group is also responsible for monitoring the metrics and diagnosing the causes for anomalies detected. His research interests are in web search and advertising since 2006. He obtained his PhD in Computer Science from the University of Illinois at Urbana-Champaign in 1999.
Kostas Tsioutsiouliklis is a principal technical lead for the content relevance group of Yahoo! Web Search. His group is responsible for defining metrics for the production content systems, focusing mainly in the crawler and web graph systems, as well as designing policies to improve crawler and index tiering performance. His research interests are in web search in general since 2001. He obtained his PhD in Computer Science from Princeton University in 2002.
Emre Velipasaoglu is a senior technical lead for the search relevance group of Yahoo! Web Search. His group is responsible for designing and developing the entire suite of ranking systems for the production web search. His research interests are in machine learning since 1991 and web search in general since 2005. He obtained his PhD in Electrical and Computer Engineering from Purdue University in 1997.
| 
