Specialized tools, such as IDEs, issue trackers, and code review tools, are an indispensable part of the modern software engineering process. These tools are constantly evolving. Besides enabling tools to support a wider range of technologies and frameworks, we are learning to provide additional features in completely new ways. One prominent stream of innovation in software engineering tools is dedicated to utilizing historical data to enable data-driven features, such as defect prediction engines and recommender systems, which leverage records of prior activity to assist with decision making. Many data-driven features in software engineering tools initially get born out of the context of real-world tools as techniques devised and evaluated in synthetic settings by researchers. While convenient, synthetic evaluation of approaches that are ultimately aimed at bringing improvement to real world problems involves a number of simplifications and assumptions. In this dissertation, we highlight several aspects that, while vital for bringing innovative methods to software engineering tools, are often discarded in existing research. We closely explore several topics specific to artificial evaluation environments, such as simplifications in mining file modification histories, use of synthetic datasets for source code authorship attribution, and a gap between accuracy of reviewer recommendation models and their perception by users. Moreover, we make a case for sharing technical artifacts by converting data mining pipelines into reusable tools, and propose a novel approach to modeling expertise transfer from code modification by capturing individual contribution style of developers. Key contributions of this dissertation include a high-level model of the lifecycle of a data-driven software engineering technique, a discussion of dangerous assumptions and simplifications that are made on every step in this lifecycle, a demonstration of importance of a careful approach to mining software repositories, and a demonstration of serious misalignment between artificial evaluation and realistic environments for the problems of code reviewer recommendation and code authorship attribution. We conclude the dissertation by discussing underlying reasons for misalignment between research environments and real-world tools, and propose potential steps to narrow it down and ultimately accelerate innovation in software engineering tooling.

Automated testing has become an essential process for improving the quality of software systems. Automated tests can help ensure that production code is robust under many usage conditions and that code meets performance and security needs. Nevertheless, writing effective tests is challenging and, unfortunately, often neglected. In the first part of this dissertation, we summarize and explain why test and production code are not treated with the same care, and we set our goal: we want to discover new techniques and tools to support developers when writing and reviewing test code. To this aim, we investigate the impact of test design issues on code quality and the practices when writing and reviewing test code.

First, we create and make publicly available Pydriller, a Python framework to analyze software repositories that will help us gather important information for the following studies. Then, we study test design issues and their impact on the overall software code quality, demonstrating how important it is to have a good and effective test suite. Afterward, together with SIG, a company setting in which part of this dissertation was conducted, we study how developers in industry react to these test design issues. Our results show that the current detection rules for test issues are not precise enough and, more importantly, do not support prioritization. We present new rules that can be used to prioritize these issues and show that the results achieved with the new rules better align with developers' perception of importance. In the second part of the dissertation we focus on how to help developers better reviewing test code. First, we investigate developers' needs when it comes to code reviewing, identifying seven high-level information needs that could be addressed through automated tools, saving up time for reviewers. Then, we focus on code review of test code specifically: we first study when and how developers review test code, identifying current practices, revealing the challenges faced during test code reviews, and uncovering needs for tools that can support the review of test code. Later, we investigate the impact of Test-Driven Code Review (TDR) on the code review effectiveness, showing that it can increase the number of test code issues found. We discuss when TDR can and can not be applied and why not all developers see TDR as a worthy practice.

Code review is a widely used technique to support software quality. It is a manual activity, often subject to repetitive and tedious tasks that increase the mental load of reviewers and compromise their effectiveness. The developer-centered nature of code review can represent a bottleneck that does not scale in large systems with the consequence of com- promising firms’ profits. This challenge has led to an entire line of research on code review improvement.
In this thesis, we present our results and remarks on the effectiveness of using fine- grained defect prediction in code review while investigating what are the information needs that lead a proper code review. We started reimplementing the state of the art of defect prediction to understand its replicability; then, we evaluated this model in a more realistic scenario that is typically considered. To improve defect prediction techniques, we come up with a fine-grained just-in-time defect prediction model that anticipates the prediction at commit time and reduces the granularity at the file level. After that, we explored how to improve further prediction performance by using alternative sources of information. We conducted a comprehensive investigation of code comments written by both open and closed source developers. Finally, to understand how to improve code review further, we explored from a reviewers’ perspective what is the information that reviewers need to lead a proper code review.
Our findings show that the state of the art of defect prediction, when evaluated in a realistic scenario, cannot be directly used to support code review. Furthermore, we assessed that alternative sets of metrics, anticipated feedback, and fine-grained suggestions represent independent directions to improve prediction performance. Finally, we discovered that research must create intelligent tools that other than predict defects must satisfy actual reviewers’ needs, such as expert selection, splittable changes, realtime communication, and self summarization of changes.

Code reviews are a widely adopted practice in software engineering that is proven to increase the quality of the code. Despite its evolution in the last decade, it still presents a number of challenges, such as understanding the changeset in review. In this thesis we research the usage of Software Visualisation paradigms to aid reviewers in the change-understanding process with a tool-based approach. Based on a survey of the code change visualisation and code navigation research areas, we devise a set of candidate prototypes of a cognitive support review tool, which we iteratively refine involving developers in the process. Through an online survey, we select one of them and build CHANGEVIZ, the implementation of our novel code review environment. The effectiveness of our approach is validated with a preliminary experiment in which developers perform change-review tasks in our review environment. The results suggest that the features incorporated by our tool are valuable for reviewers.

Code review has been proved to be an extremely important practice to ensure softwarequality. In recent years, the trend has moved towards modern code review, a lightweight and less strict paradigm. Despite its many advantages, this approach still has room for further improvement, especially in the area of cognitive support for reviewers.
Previous research stated how ordering code changes based on their relations may constitute an effective way to support reviewers. Based on this premise, this work focuses on studying how this ordering theory may be applied in practice. As result, a tool that automatically orders the modifications in a commit has been created.
Moreover, the tool has been tested and an initial investigation of the perceived
usefulness of its relations has been conducted. Finally, it has been investigated if the ordering produced by the tool is identified as useful by the developers and which factors may influence this choice.

Agile software development has interested researchers for the last decade. Agile software development teams develop iteration sessions that often last weeks. During development, teams work on technical code and its content. Intuitively, more effort is required to implement new features in poorly constructed code with low quality. This study investigates if and how developers consider the quality of their code during their agile effort estimations. Furthermore, we investigate whether the accuracy of their estimations could increase if developers considered the quality of the code. This study is conducted in a large software development department, that is part of Royal Dutch Shell. We take a mixed method approach, where we interview nine developers and quality experts and mine the repositories of six agile development teams. Initially, we reviewed the existing importance measures of code quality during effort estimations, including how code quality is maintained. We also evaluate the impact of code quality on estimation accuracy.

Developers did not consider code quality high on the priority list during the estimation stage of development. Similarly, we did not find an empirical relationship between the quality metrics and effort estimations. Surprisingly, code quality only had minor effects on the accuracy of the effort estimations. Developers did often encounter quality issues in legacy code. However, overall our study shows that code quality is only of minor importance during agile effort estimations.