Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar—all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives, and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.@en

The paper discusses some trends in end user programming (EUP) and takes inspiration from the discussions in a panel and in a vertical session on research evaluation within the second Search Computing workshop. We discuss the controversial successes and failures in this field and we elaborate on which facilities could foster adoption of end user programming. We discuss various dimensions of end user programming, including vertical versus horizontal language definition, declarative versus imperative approaches. We exemplify our discussion in the realistic scenario of research evaluation by comparing the Search Computing and ResEval approaches.@en