How can crowdsourced workers effectively rate artwork images produced by Generative Adversarial Network transformers?

Abstract

Generative Adversarial Networks (GANs) can create artwork images and we need effective ways of rating their aesthetic values. This could help us determine the most aesthetic artwork images (and identify the GANs that created them) and train GANs to produce more aesthetic artwork images in the future. In this research, we analyzed the effectiveness of using two different survey formats (binary-choice and four-choice) for displaying GAN-produced artwork images to crowd- sourced workers and gathering their ratings. The artwork images were of different landscapes like the desert, arctic, coastal regions, etc. Additionally, we investigated how the choice of showing different images together (image groupings) per question affects the final rating results. Results demonstrate that the four-choice format is superior to the binary-choice format in producing more consistent, reliable, and accurate results. The effects of the different image groupings were insignif- icant for the results of the four-choice format. In contrast, different image groupings displayed statistically significant changes in the results for the binary-choice format. However, it was found that crowdsourced workers preferred the binary-choice format more as they found it to be less strenuous and more effective in allowing them to express their rating choices.