Evaluating the Quality of Opponent Models in Automated Bilateral Negotiations

Abstract

Automated negotiation agents are agents that interact in an environment for the settlement of a mutual concern. An important factor influencing the performance of a negotiation agent is how it takes the opponent into account. The main challenge in this aspect, is that opponents typically hide private information to avoid exploitation. In such a setting, an opponent model can help by estimating the opponent's strategy or preference profile. This work contains the first recent survey of opponent models in automated negotiation. One of the main conclusions of this survey, is that currently there is no fair method to evaluate and compare the quality of a set of opponent models. Insight in the quality of an opponent model could lead to the development of a better model. In this work we focus on a specific type of opponent models which model the opponent's preferences. Based on a detailed analysis of the factors influencing the quality of this type of opponent model, we introduce and apply two fair measurement methods to quantify the performance gain relative to not using an opponent model and the accuracy of the model. Our contribution to the field of automated negotiation is threefold; first, we provide a comprehensive survey of opponent models; second, we introduce a method to isolate the components of a negotiation strategy; finally, we construct and apply two fair evaluation methods to quantify the quality of a set of opponent models which model the opponent's preferences. Taken together, this work structures the field of opponent models and provides insight in how to improve existing models.