From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

Journal article (2019)

Authors

Nicola Ferro Università degli Studi di Padova
Norbert Fuhr Universität Duisburg-Essen
Gregory Grefenstette Florida Institute for Human and Machine Cognition (IHMC)
Tsvi Kuflik University of Haifa
Krister Lindén University of Helsinki
Bernardo Magnini Trento Institute for Fundamental Physics and Application
Jian-Yun Nie University of Quebec
N. Tintarev Web Information Systems - EEMCS
undefined More Authors More Authors External organisation
Research Group
Web Information Systems (EEMCS) (TU Delft)
Copyright: © 2019 Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, N. Tintarev, More Authors
DOI: https://doi.org/10.4230/DagMan.7.1.96
More Info
expand_more

Published Date

2019

Language

English

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Web Information Systems

Abstract

We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.