Comparing Quantitative Metrics for Generative Adversarial Neural Networks

Comparing Quantitative Metrics for Generative Adversarial Neural Networks

Slangewal, Bart (TU Delft Electrical Engineering, Mathematics and Computer Science)

Tax, David (mentor)

Delft University of Technology

2019-07-01

Since their conception in 2014, a large number of Generative Adversarial Networks (GANs) [2] has been pro- posed and developed. GANs have achieved great results in realistic image generation, among other fields. Recently, stunning images have been produced. The theory and application of GANs has received much attention. However, the evaluation of these models has not been studied nearly as extensively. GANs are often evaluated by visual inspection. This is a time consuming process, that inherently suffers from being subjective. There has been research into quantitative metrics, which can be automated. There has even been some research into the relative merits of such metrics. [1][9][4]. However, the question which metric is the most suitable for evaluating GANs in the field of realistic image generation remains open. It is important that consensus is reached. If there is no agreed upon method of objectively measuring progress, it is hard to say which techniques are effective. This impedes the entire field of study. This paper hopes to contribute to answering this question by applying a variety of proposed metrics to a variety of different GANs. This way, it is possible to tell which metrics agree with each other, and which rate GANs differently. If all, or many, of the likely metrics proposed in earlier work agree with each other, this is a good sign that they are rating GANs objectively. The results of this experiment are combined with a very brief survey of earlier work, in order to recommend some quantitative metrics for rating GANs.

GAN
GAN evaluation
Generative Adversarial Network
Quantitative metrics
Quantitative analysis

http://resolver.tudelft.nl/uuid:8702a381-90f8-4a02-9833-1a4a5b02d097

Student theses

bachelor thesis

© 2019 Bart Slangewal