AS
András Serény
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
3 records found
1
Idomaar
A Framework for Multi-dimensional Benchmarking of Recommender Algorithms
Conference paper
(2016)
-
Mario Scriminaci, Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Davide Malagoli, András Serény, Till Plumbaum
In real-world scenarios, recommenders face non-functional requirementsof technical nature and must handle dynamic data in the formof sequential streams. Evaluation of recommender systems musttake these issues into account in order to be maximally informative.In this paper, we present Idomaar—a framework that enables theefficient multi-dimensional benchmarking of recommender algorithms.Idomaar goes beyond current academic research practicesby creating a realistic evaluation environment and computing botheffectiveness and technical metrics for stream-based as well as setbasedevaluation. A scenario focussing on “research to prototypingto productization” cycle at a company illustrates Idomaar’s potential.We show that Idomaar simplifies testing with varying configurationsand supports flexible integration of different data.
...
In real-world scenarios, recommenders face non-functional requirementsof technical nature and must handle dynamic data in the formof sequential streams. Evaluation of recommender systems musttake these issues into account in order to be maximally informative.In this paper, we present Idomaar—a framework that enables theefficient multi-dimensional benchmarking of recommender algorithms.Idomaar goes beyond current academic research practicesby creating a realistic evaluation environment and computing botheffectiveness and technical metrics for stream-based as well as setbasedevaluation. A scenario focussing on “research to prototypingto productization” cycle at a company illustrates Idomaar’s potential.We show that Idomaar simplifies testing with varying configurationsand supports flexible integration of different data.
Overview of newsreel’16
Multi-dimensional evaluation of real-time stream-recommendation algorithms
Conference paper
(2016)
-
Benjamin Kille, Andreas Lommatzsch, Gebrekirstos G Gebremeskel, Frank Hopfgartner, Martha Larson, Jonas Seiler, Davide Malagoli, András Serény, Torben Brodt, Arjen P De Vries
Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.
...
Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.
Benchmarking News Recommendations
The CLEF NewsREEL Use Case
Journal article
(2015)
-
Frank Hopfgartner, Torben Brodt, Jonas Seiler, Benjamin Kille, Andreas Lommatzsch, Martha Larson, Roberto Turrin, András Serény
The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms. The goal is to create an algorithm that is able to generate news items that users would click, respecting a strict time constraint. The lab challenges participants to compete in either a “living lab” (Task 1) or perform an evaluation that replays recorded streams (Task 2). In this report, we discuss the objectives and challenges of the NewsREEL lab, summarize last year’s campaign and outline the main research challenges that can be addressed by participating in NewsREEL 2016.
...
The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to evaluate and optimize news recommender algorithms. The goal is to create an algorithm that is able to generate news items that users would click, respecting a strict time constraint. The lab challenges participants to compete in either a “living lab” (Task 1) or perform an evaluation that replays recorded streams (Task 2). In this report, we discuss the objectives and challenges of the NewsREEL lab, summarize last year’s campaign and outline the main research challenges that can be addressed by participating in NewsREEL 2016.