Context: Automated classifiers, often based on machine learning (ML), are increasingly used in software engineering (SE) for labelling previously unseen SE data. Researchers have proposed automated classifiers that predict if a code chunk is a clone, if a requirement is functional or non-functional, if the outcome of a test case is non-deterministic, etc. Objective: The lack of guidelines for applying and reporting classification techniques for SE research leads to studies in which important research steps may be skipped, key findings might not be identified and shared, and the readers may find reported results (e.g., precision or recall above 90%) that are not a credible representation of the performance in operational contexts. The goal of this paper is to advance ML4SE research by proposing rigorous ways of conducting and reporting research. Results: We introduce the ECSER (Evaluating Classifiers in Software Engineering Research) pipeline, which includes a series of steps for conducting and evaluating automated classification research in SE. Then, we conduct two replication studies where we apply ECSER to recent research in requirements engineering and in software testing. Conclusions: In addition to demonstrating the applicability of the pipeline, the replication studies demonstrate ECSER’s usefulness: not only do we confirm and strengthen some findings identified by the original authors, but we also discover additional ones. Some of these findings contradict the original ones.@en

Socially Assistive Robots (SARs) are increasingly used in dementia and elderly care. In order to provide effective assistance, SARs need to be personalized to individual patients and account for stimulating their divergent thinking in creative ways. Rule-based fuzzy logic systems provide effective methods for automated decision-making of SARs. However, expanding and modifying the rules of fuzzy logic systems to account for the evolving needs, preferences, and medical conditions of patients can be tedious and costly. In this paper, we introduce EFS4SAR, a novel Evolving Fuzzy logic System for Socially Assistive Robots that supports autonomous evolution of the fuzzy rules that steer the behavior of the SAR. EFS4SAR combines traditional rule-based fuzzy logic systems with evolutionary algorithms, which model the process of evolution in nature and have shown to result in creative behaviors. We evaluate EFS4SAR via computer simulations on both synthetic and real-world data. The results show that the fuzzy rules evolved over time are not only personalized with respect to the personal preferences and therapeutic needs of the patients, but they also meet the following criteria for creativity of SARs: originality and effectiveness of the therapeutic tasks proposed to the patients. Compared to existing evolving fuzzy systems, EFS4SAR achieves similar effectiveness with higher degree of originality.@en

Simulation is a useful tool for evaluating behavioral interventions when the adoption rate among a population is uncertain. Individual agent models are often prohibitively expensive, but, unlike stochastic models, allow studying compliance heterogeneity. In this paper we demonstrate the feasibility of evaluating behavioral intervention policies using large-scale data-driven agent-based simulations. We explain how the simulation is calibrated with respect to real-world data, and demonstrate the utility of our approach by studying the effectiveness of interventions used in Virginia in early 2020 through counterfactual simulations.@en

In multi-agent systems, norm enforcement is a mechanism for steering the behavior of individual agents in order to achieve desired system-level objectives. Due to the dynamics of multi-agent systems, however, it is hard to design norms that guarantee the achievement of the objectives in every operating context. Also, these objectives may change over time, thereby making previously defined norms ineffective. In this paper, we investigate the use of system execution data to automatically synthesise and revise conditional prohibitions with deadlines, a type of norms aimed at prohibiting agents from exhibiting certain patterns of behaviors. We propose DDNR (Data-Driven Norm Revision), a data-driven approach to norm revision that synthesises revised norms with respect to a data set of traces describing the behavior of the agents in the system. We evaluate DDNR using a state-of-the-art, off-the-shelf urban traffic simulator. The results show that DDNR synthesises revised norms that are significantly more accurate than the original norms in distinguishing adequate and inadequate behaviors for the achievement of the system-level objectives. @en

Norms have been widely proposed as a way of coordinating and controlling the activities of agents in a multi-agent system (MAS). A norm specifies the behaviour an agent should follow in order to achieve the objective of the MAS. However, designing norms to achieve a particular system objective can be difficult, particularly when there is no direct link between the language in which the system objective is stated and the language in which the norms can be expressed. In this paper, we consider the problem of synthesising a norm from traces of agent behaviour, where each trace is labelled with whether the behaviour satisfies the system objective. We show that the norm synthesis problem and several related problems are NP-complete.@en

In multi-agent systems, norm enforcement is a mechanism for steering the behavior of individual agents in order to achieve desired system-level objectives. Due to the dynamics of multi-agent systems, however, it is hard to design norms that guarantee the achievement of the objectives in every operating context. Also, these objectives may change over time, thereby making previously defined norms ineffective. In this paper, we investigate the use of system execution data to automatically synthesise and revise conditional prohibitions with deadlines, a type of norms aimed at preventing agents from exhibiting certain patterns of behaviors. We propose DDNR (Data-Driven Norm Revision), a data-driven approach to norm revision that synthesises revised norms with respect to a data set of traces describing the behavior of the agents in the system. We evaluate DDNR using a state-of-the-art, off-the-shelf urban traffic simulator. The results show that DDNR synthesises revised norms that are significantly more accurate than the original norms in distinguishing adequate and inadequate behaviors for the achievement of the system-level objectives.@en