From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)
Nicola Ferro (Università degli Studi di Padova)
Norbert Fuhr (Universität Duisburg-Essen)
Gregory Grefenstette (Florida Institute for Human and Machine Cognition (IHMC))
Tsvi Kuflik (University of Haifa)
Krister Lindén (University of Helsinki)
Bernardo Magnini (Trento Institute for Fundamental Physics and Applications)
Jian-Yun Nie (University of Quebec)
Raffaele Perego (Consiglio Nazionale delle Ricerche (CNR))
Nava Tintarev (TU Delft - Web Information Systems)
undefined More Authors (External organisation)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.
help_outline