Monitoring-aware IDEs

Abstract

It has become a common practice for software developers to analyse monitoring data as a means to understand issues, trends and performance of large-scale software systems. The combination of this practice with the software development workflow can however be cumbersome at times. One recent effort into evaluating the effects of integrating monitoring directly into the software development workflow has been done at the large-scale payment company Adyen in cooperation with the TU Delft. This project is a new attempt at making a flexible, open-source and useful monitoring tool based on the results of that research effort. This report will detail how the system is designed, implemented and the development methodology that was used for its creation. After researching popular monitoring stacks and solutions, an architecture for the Hyperion pipeline was designed that would maximize compatibility with logging setups. The monitoring data is first retrieved from a log source like Logstash or Elasticsearch. It is preprocessed by a pipeline consisting of configurable and flexible plugins. A large range of plugins is supported: functionalities include debugging, transformation, load balancing, and adding new data from third-party systems. There is also extensive support for third-party developers to create their own pipeline plugin to manipulate data in any way required, which allows Hyperion to account for unconventional logging setups. The pipeline typically ends at some type of aggregator, which exposes an interface for developer tools to retrieve metrics. These metrics can be queried with the Hyperion IDE plugin, which visualizes them in the development environment itself. To verify that the designed Hyperion pipeline adheres to the requirements defined in the research report, validation tests have been performed on every module of the system. All code is extensively tested with unit, integration, and container-based system tests. An evaluation of the performance of the pipeline additionally verified the scalability of the system even when provided with a large number of incoming logs. Finally, in order to ensure easy extension and adoption of the Hyperion pipeline a large amount of documentation was written, that explains the architecture of the project and how a third-party developer can adjust and extend it.