Identifying and Managing Technical Debt in Complex Distributed Systems

Abstract

The term technical debt has been used to described the increased cost of changing or maintaining a system due to expedient shortcuts taken during development, possibly due to financial or time constraints. The term has gained significant attention in software engineering research and the agile community. Tribler, a platform to share and discover content in a complete decentralized way, has accumulated a tremendous amount of technical debt over the last ten years of scientific research in the area of peer-to-peer networking. The platform suffers from a complex architecture, an unintuitive user interface, an incomplete and unstable testing framework and a large amount of unmaintained code. A new layered, flexible and component-based architecture that readies Tribler for the next decade of research is proposed and discussed. We lay the foundations for this new architecture by implementing a RESTful API and a new graphical user interface. By removing the old user interface, the amount of technical debt in Tribler is dramatically reduced. Additional work includes the pay off of various kind of technical debt by the means of major improvements to the testing framework, heavy modifications within the core of Tribler and changes in the infrastructure to make it more usable and robust. With the deletion of 12.581 lines, the modification of 765 lines and addition of 12.429 lines, we show that several important software metrics are improved and that we paid off a huge amount of technical debt. Raising awareness about technical debt in general is of uttermost importance if we wish to prevent deterioration of the system. Together with a code review policy and static code analysis tools to track code coverage and the amount of code violations, we hope to prevent huge amounts of technical debt in the future. We perform experiments to verify the stability and performance of various components in the core of Tribler and propose future work for the components that require more work. In our final experiment, we will test Tribler on a large scale and lay the foundations for an application testing framework that is useful for failure identification.